2024-02-21T23:01:23.483 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-21T23:01:23.483 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021 --verbose -- /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/orig.config.yaml 2024-02-21T23:01:23.528 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T23:01:23.614 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021 branch: wip-lusov-qdb description: fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} email: null first_in_suite: false job_id: '7570021' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mds beacon grace: 60 mds heartbeat grace: 60 osd heartbeat grace: 60 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds valgrind exit: true mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd crush smoke test: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow request - SLOW_OPS - MON_DOWN sha1: 3bfc837d7b45ce8e095d92fa9c342964cea5e95f valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes watchdog: daemon_restart: normal ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: valgrind: - --tool=memcheck - --leak-check=full - --show-reachable=yes install: ceph: debuginfo: true flavor: default sha1: 3bfc837d7b45ce8e095d92fa9c342964cea5e95f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 3bfc837d7b45ce8e095d92fa9c342964cea5e95f owner: scheduled_leonidus@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 6562 sha1: 3bfc837d7b45ce8e095d92fa9c342964cea5e95f sleep_before_teardown: 0 subset: 111/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3bfc837d7b45ce8e095d92fa9c342964cea5e95f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIQO4F2tzpA2TGhD0+Q5J562sN4OU5mir5R/JLR2UmtSabM3oZhxjURS0e0nm69nVx0/HmIPnB93HuV8bXkZtqM= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIY6+OhPMMUUKiclL98O2Lt81V/BXxVGzuWGo/6K6ZtYxlt+DbvCAGa34duU8PucAHFMXB8TFlHfqV/kqyG528E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-21_22:31:03 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-21T23:01:23.615 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa; will attempt to use it 2024-02-21T23:01:23.615 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks 2024-02-21T23:01:23.616 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-21T23:01:23.617 INFO:teuthology.task.internal:Checking packages... 2024-02-21T23:01:23.641 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3bfc837d7b45ce8e095d92fa9c342964cea5e95f' 2024-02-21T23:01:23.641 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-21T23:01:23.641 INFO:teuthology.packaging:ref: None 2024-02-21T23:01:23.641 INFO:teuthology.packaging:tag: None 2024-02-21T23:01:23.641 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-02-21T23:01:23.641 INFO:teuthology.packaging:sha1: 3bfc837d7b45ce8e095d92fa9c342964cea5e95f 2024-02-21T23:01:23.642 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-02-21T23:01:23.862 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1643.g3bfc837d 2024-02-21T23:01:23.863 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-21T23:01:23.870 INFO:teuthology.task.internal:no buildpackages task found 2024-02-21T23:01:23.870 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-21T23:01:23.893 INFO:teuthology.task.internal:Saving configuration 2024-02-21T23:01:23.910 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-21T23:01:23.917 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-21T23:01:23.940 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-21 22:56:29.581974', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIQO4F2tzpA2TGhD0+Q5J562sN4OU5mir5R/JLR2UmtSabM3oZhxjURS0e0nm69nVx0/HmIPnB93HuV8bXkZtqM='} 2024-02-21T23:01:23.959 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-21 22:56:29.583097', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIY6+OhPMMUUKiclL98O2Lt81V/BXxVGzuWGo/6K6ZtYxlt+DbvCAGa34duU8PucAHFMXB8TFlHfqV/kqyG528E='} 2024-02-21T23:01:23.959 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-21T23:01:23.965 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-02-21T23:01:23.966 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-02-21T23:01:23.966 INFO:teuthology.run_tasks:Running task console_log... 2024-02-21T23:01:24.056 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa4c15c0f70>, signals=[15]) 2024-02-21T23:01:24.057 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-21T23:01:24.063 INFO:teuthology.task.internal:Opening connections... 2024-02-21T23:01:24.063 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-02-21T23:01:24.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T23:01:24.156 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-02-21T23:01:24.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T23:01:24.247 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-21T23:01:24.253 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-02-21T23:01:24.271 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-02-21T23:01:24.272 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-02-21T23:01:24.328 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-21T23:01:24.329 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-21T23:01:24.330 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-02-21T23:01:24.347 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-02-21T23:01:24.364 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-02-21T23:01:24.365 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-02-21T23:01:24.421 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-02-21T23:01:24.421 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-02-21T23:01:24.421 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-02-21T23:01:24.421 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-21T23:01:24.422 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-21T23:01:24.422 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-02-21T23:01:24.446 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-21T23:01:24.456 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-21T23:01:24.476 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-21T23:01:24.476 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-02-21T23:01:24.479 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-02-21T23:01:24.495 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-21T23:01:24.501 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-21T23:01:24.501 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-02-21T23:01:24.536 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-02-21T23:01:24.580 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-21T23:01:24.632 INFO:teuthology.run_tasks:Running task kernel... 2024-02-21T23:01:24.648 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-21T23:01:24.648 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-21T23:01:24.648 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-21T23:01:24.649 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-21T23:01:24.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-21T23:01:24.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-21T23:01:24.666 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-02-21T23:01:24.667 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-02-21T23:01:24.723 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-361.el9.x86_64 2024-02-21T23:01:24.723 INFO:teuthology.task.kernel:Running kernel on smithi142: 5.14.0-361.el9.x86_64 2024-02-21T23:01:24.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum install -y kernel 2024-02-21T23:01:24.724 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-361.el9.x86_64 2024-02-21T23:01:24.724 INFO:teuthology.task.kernel:Running kernel on smithi133: 5.14.0-361.el9.x86_64 2024-02-21T23:01:24.724 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum install -y kernel 2024-02-21T23:01:27.072 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - AppStream 11 MB/s | 8.1 MB 00:00 2024-02-21T23:01:27.305 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - AppStream 8.2 MB/s | 8.1 MB 00:00 2024-02-21T23:01:29.069 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.5 MB 00:00 2024-02-21T23:01:29.230 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.5 MB 00:00 2024-02-21T23:01:30.304 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-02-21T23:01:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-02-21T23:01:40.050 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 754 kB/s | 8.0 MB 00:10 2024-02-21T23:01:49.395 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 458 kB/s | 6.1 MB 00:13 2024-02-21T23:01:50.896 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 15 kB 00:00 2024-02-21T23:01:53.972 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 20 MB 00:02 2024-02-21T23:01:59.071 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-21T23:02:00.124 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:18 2024-02-21T23:02:01.341 INFO:teuthology.orchestra.run.smithi142.stderr:Modular dependency problems: 2024-02-21T23:02:01.341 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-02-21T23:02:01.341 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 1: conflicting requests 2024-02-21T23:02:01.341 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.341 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 2: conflicting requests 2024-02-21T23:02:01.341 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.341 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 3: conflicting requests 2024-02-21T23:02:01.341 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 4: conflicting requests 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 5: conflicting requests 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 6: conflicting requests 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 7: conflicting requests 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 8: conflicting requests 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 9: conflicting requests 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 10: conflicting requests 2024-02-21T23:02:01.342 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 11: conflicting requests 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 12: conflicting requests 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 13: conflicting requests 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 14: conflicting requests 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 15: conflicting requests 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 16: conflicting requests 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 17: conflicting requests 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.343 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 18: conflicting requests 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 19: conflicting requests 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 20: conflicting requests 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 21: conflicting requests 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 22: conflicting requests 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 23: conflicting requests 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 24: conflicting requests 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.344 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 25: conflicting requests 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 26: conflicting requests 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 27: conflicting requests 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 28: conflicting requests 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 29: conflicting requests 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 30: conflicting requests 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 31: conflicting requests 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 32: conflicting requests 2024-02-21T23:02:01.345 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 33: conflicting requests 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 34: conflicting requests 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 35: conflicting requests 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 36: conflicting requests 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 37: cannot install the best candidate for the job 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.346 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 38: conflicting requests 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 39: cannot install the best candidate for the job 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 40: cannot install the best candidate for the job 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.347 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 41: cannot install the best candidate for the job 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 42: cannot install the best candidate for the job 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 43: cannot install the best candidate for the job 2024-02-21T23:02:01.348 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.349 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.349 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.349 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.349 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 44: cannot install the best candidate for the job 2024-02-21T23:02:01.349 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.349 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.349 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.349 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-21T23:02:01.761 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-21T23:02:01.762 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-21T23:02:01.861 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-02-21T23:02:01.873 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-21T23:02:01.873 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-02-21T23:02:01.873 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout: kernel x86_64 5.14.0-419.el9 baseos 6.0 M 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core x86_64 5.14.0-419.el9 baseos 21 M 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules x86_64 5.14.0-419.el9 baseos 39 M 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core x86_64 5.14.0-419.el9 baseos 33 M 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout:Install 4 Packages 2024-02-21T23:02:01.874 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:02:01.875 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 99 M 2024-02-21T23:02:01.875 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 122 M 2024-02-21T23:02:01.875 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-02-21T23:02:02.589 INFO:teuthology.orchestra.run.smithi142.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 9.0 MB/s | 6.0 MB 00:00 2024-02-21T23:02:03.056 INFO:teuthology.orchestra.run.smithi142.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 18 MB/s | 21 MB 00:01 2024-02-21T23:02:03.906 INFO:teuthology.orchestra.run.smithi142.stdout:(3/4): kernel-modules-core-5.14.0-419.el9.x86_6 25 MB/s | 33 MB 00:01 2024-02-21T23:02:04.314 INFO:teuthology.orchestra.run.smithi142.stdout:(4/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 16 MB/s | 39 MB 00:02 2024-02-21T23:02:04.318 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-02-21T23:02:04.318 INFO:teuthology.orchestra.run.smithi142.stdout:Total 41 MB/s | 99 MB 00:02 2024-02-21T23:02:04.391 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-02-21T23:02:05.214 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-02-21T23:02:05.576 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-02-21T23:02:05.576 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-02-21T23:02:06.141 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 15 kB 00:00 2024-02-21T23:02:06.936 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-02-21T23:02:06.937 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-02-21T23:02:07.793 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2024-02-21T23:02:09.864 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-02-21T23:02:10.511 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-21T23:02:10.630 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-21T23:02:12.775 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-21T23:02:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-21T23:02:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-21T23:02:15.018 INFO:teuthology.orchestra.run.smithi133.stderr:Modular dependency problems: 2024-02-21T23:02:15.018 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-02-21T23:02:15.018 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 1: conflicting requests 2024-02-21T23:02:15.018 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.018 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 2: conflicting requests 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 3: conflicting requests 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 4: conflicting requests 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 5: conflicting requests 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 6: conflicting requests 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 7: conflicting requests 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 8: conflicting requests 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 9: conflicting requests 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.019 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 10: conflicting requests 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 11: conflicting requests 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 12: conflicting requests 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 13: conflicting requests 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 14: conflicting requests 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 15: conflicting requests 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 16: conflicting requests 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.020 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 17: conflicting requests 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 18: conflicting requests 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 19: conflicting requests 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 20: conflicting requests 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 21: conflicting requests 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 22: conflicting requests 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.021 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 23: conflicting requests 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 24: conflicting requests 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 25: conflicting requests 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 26: conflicting requests 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 27: conflicting requests 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 28: conflicting requests 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 29: conflicting requests 2024-02-21T23:02:15.022 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.023 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 30: conflicting requests 2024-02-21T23:02:15.023 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.023 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 31: conflicting requests 2024-02-21T23:02:15.023 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.023 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 32: conflicting requests 2024-02-21T23:02:15.023 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.023 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 33: conflicting requests 2024-02-21T23:02:15.023 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.023 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 34: conflicting requests 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 35: conflicting requests 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 36: conflicting requests 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 37: cannot install the best candidate for the job 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 38: conflicting requests 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.024 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 39: cannot install the best candidate for the job 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 40: cannot install the best candidate for the job 2024-02-21T23:02:15.025 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 41: cannot install the best candidate for the job 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 42: cannot install the best candidate for the job 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 43: cannot install the best candidate for the job 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.026 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.027 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.027 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.027 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 44: cannot install the best candidate for the job 2024-02-21T23:02:15.027 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.027 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.027 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.027 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-21T23:02:15.466 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-21T23:02:15.466 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-21T23:02:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-02-21T23:02:15.592 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-02-21T23:02:15.592 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2024-02-21T23:02:15.592 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-02-21T23:02:15.592 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-02-21T23:02:15.592 INFO:teuthology.orchestra.run.smithi133.stdout: kernel x86_64 5.14.0-419.el9 baseos 6.0 M 2024-02-21T23:02:15.592 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-02-21T23:02:15.592 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-core x86_64 5.14.0-419.el9 baseos 21 M 2024-02-21T23:02:15.593 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules x86_64 5.14.0-419.el9 baseos 39 M 2024-02-21T23:02:15.593 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-core x86_64 5.14.0-419.el9 baseos 33 M 2024-02-21T23:02:15.593 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:02:15.593 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-02-21T23:02:15.593 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-02-21T23:02:15.593 INFO:teuthology.orchestra.run.smithi133.stdout:Install 4 Packages 2024-02-21T23:02:15.593 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:02:15.594 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 99 M 2024-02-21T23:02:15.594 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 122 M 2024-02-21T23:02:15.594 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-02-21T23:02:17.135 INFO:teuthology.orchestra.run.smithi133.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 4.1 MB/s | 6.0 MB 00:01 2024-02-21T23:02:18.385 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-21T23:02:18.593 INFO:teuthology.orchestra.run.smithi133.stdout:(2/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 13 MB/s | 39 MB 00:02 2024-02-21T23:02:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:02:18.794 INFO:teuthology.orchestra.run.smithi133.stdout:(3/4): kernel-core-5.14.0-419.el9.x86_64.rpm 6.6 MB/s | 21 MB 00:03 2024-02-21T23:02:23.394 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:02:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 4.8 MB/s | 33 MB 00:06 2024-02-21T23:02:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-02-21T23:02:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:Total 12 MB/s | 99 MB 00:08 2024-02-21T23:02:25.009 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-02-21T23:02:25.387 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-02-21T23:02:25.387 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-02-21T23:02:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-02-21T23:02:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-02-21T23:02:29.964 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-02-21T23:02:30.608 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-21T23:02:30.743 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-21T23:02:33.536 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-21T23:02:33.719 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-21T23:02:38.437 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-21T23:02:38.696 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:02:43.410 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:02:52.532 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:02:52.532 INFO:teuthology.orchestra.run.smithi142.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-21T23:02:52.532 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:02:52.617 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:02:53.077 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:02:53.077 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-21T23:02:53.077 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-21T23:02:53.077 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-21T23:02:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:02:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:02:53.524 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-02-21T23:02:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-21T23:02:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-21T23:02:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-21T23:02:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-21T23:02:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:02:53.524 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-02-21T23:02:54.102 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-02-21T23:02:54.260 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-21T23:02:54.260 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-21T23:02:54.260 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-21T23:02:54.260 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-21T23:02:54.260 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-21T23:02:54.261 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-21T23:02:54.261 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=None, version=distro) 2024-02-21T23:02:54.261 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-02-21T23:02:54.382 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-21T23:02:54.382 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-21T23:02:54.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2024-02-21T23:02:54.424 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2024-02-21T23:02:54.424 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2024-02-21T23:02:54.424 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.06 2024-02-21T23:02:54.424 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 68.el9 2024-02-21T23:02:54.424 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2024-02-21T23:02:54.424 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-21T23:02:54.424 INFO:teuthology.orchestra.run.smithi142.stdout:Group : Unspecified 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 8264254 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : builder@centos.org 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : CentOS 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-21T23:02:54.425 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-21T23:02:54.426 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2024-02-21T23:02:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:02:54.426 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2024-02-21T23:02:54.426 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-21T23:02:54.426 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 5.14.0-419.el9.x86_64 2024-02-21T23:02:54.427 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-21T23:02:55.070 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-02-21T23:02:56.502 INFO:teuthology.orchestra.run.smithi142.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-21T23:02:56.525 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-02-21T23:02:56.528 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-02-21T23:02:56.584 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.dGbOW7phJf 2024-02-21T23:02:56.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dGbOW7phJf 2024-02-21T23:02:56.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.dGbOW7phJf 2024-02-21T23:02:56.784 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.dGbOW7phJf is 6KB 2024-02-21T23:02:56.832 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.dGbOW7phJf 2024-02-21T23:02:56.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2024-02-21T23:02:56.910 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-21T23:02:56.910 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-21T23:02:56.911 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-21T23:02:56.911 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-21T23:02:56.912 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-21T23:02:57.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-02-21T23:03:12.614 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:03:12.614 INFO:teuthology.orchestra.run.smithi133.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-21T23:03:12.614 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:03:12.702 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:03:13.103 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:03:13.103 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-21T23:03:13.103 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-21T23:03:13.103 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-21T23:03:13.539 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-21T23:03:13.540 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:03:13.540 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-02-21T23:03:13.540 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-21T23:03:13.540 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-21T23:03:13.540 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-21T23:03:13.540 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-21T23:03:13.540 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:03:13.540 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-02-21T23:03:14.026 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q kernel | sort -rV | head -n 1 2024-02-21T23:03:14.181 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-21T23:03:14.182 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-21T23:03:14.182 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-21T23:03:14.182 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-21T23:03:14.182 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-21T23:03:14.182 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-21T23:03:14.182 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=None, version=distro) 2024-02-21T23:03:14.182 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q kernel | sort -rV | head -n 1 2024-02-21T23:03:14.304 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-21T23:03:14.305 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-21T23:03:14.305 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2024-02-21T23:03:14.342 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2024-02-21T23:03:14.342 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2024-02-21T23:03:14.342 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.06 2024-02-21T23:03:14.342 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 68.el9 2024-02-21T23:03:14.342 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2024-02-21T23:03:14.343 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-21T23:03:14.343 INFO:teuthology.orchestra.run.smithi133.stdout:Group : Unspecified 2024-02-21T23:03:14.343 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 8264254 2024-02-21T23:03:14.343 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2024-02-21T23:03:14.343 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-21T23:03:14.344 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-21T23:03:14.344 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-21T23:03:14.344 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-21T23:03:14.344 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : builder@centos.org 2024-02-21T23:03:14.344 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : CentOS 2024-02-21T23:03:14.344 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-21T23:03:14.345 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2024-02-21T23:03:14.345 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2024-02-21T23:03:14.345 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:03:14.345 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-21T23:03:14.345 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-21T23:03:14.346 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-21T23:03:14.346 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2024-02-21T23:03:14.346 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:03:14.346 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2024-02-21T23:03:14.348 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-21T23:03:14.348 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 5.14.0-419.el9.x86_64 2024-02-21T23:03:14.348 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-21T23:03:14.954 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-02-21T23:03:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-21T23:03:16.439 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-02-21T23:03:16.442 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-02-21T23:03:16.457 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.Y5qUuHbI5T 2024-02-21T23:03:16.457 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Y5qUuHbI5T 2024-02-21T23:03:16.525 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.Y5qUuHbI5T 2024-02-21T23:03:16.662 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.Y5qUuHbI5T is 6KB 2024-02-21T23:03:16.711 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.Y5qUuHbI5T 2024-02-21T23:03:16.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2024-02-21T23:03:16.790 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-21T23:03:16.790 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-21T23:03:16.790 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-21T23:03:16.790 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-21T23:03:16.791 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-21T23:03:16.958 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-02-21T23:03:27.080 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-21T23:03:27.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-21T23:03:27.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T23:03:46.966 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-21T23:03:46.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-02-21T23:03:46.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T23:04:21.564 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-02-21T23:04:27.089 DEBUG:teuthology.orchestra.remote:timed out 2024-02-21T23:04:30.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-02-21T23:04:30.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T23:04:36.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-21T23:04:36.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T23:04:36.568 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-02-21T23:04:37.019 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-21T23:04:37.020 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-419.el9.x86_64"... 2024-02-21T23:04:37.020 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-02-21T23:04:37.034 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-419.el9.x86_64 2024-02-21T23:04:37.034 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-21T23:04:37.034 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-21T23:04:37.034 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-21T23:04:38.035 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-21T23:04:38.035 DEBUG:teuthology.orchestra.run.smithi142:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-21T23:04:38.103 INFO:teuthology.orchestra.run.smithi142.stdout:ttyS1 2024-02-21T23:04:38.139 DEBUG:teuthology.parallel:result is None 2024-02-21T23:04:48.956 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-02-21T23:05:00.957 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-02-21T23:05:00.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T23:05:01.465 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-02-21T23:05:02.067 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-02-21T23:05:02.067 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-419.el9.x86_64"... 2024-02-21T23:05:02.067 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-02-21T23:05:02.081 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-419.el9.x86_64 2024-02-21T23:05:02.081 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-21T23:05:02.082 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-21T23:05:02.082 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-21T23:05:03.082 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-21T23:05:03.083 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-21T23:05:03.162 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2024-02-21T23:05:03.196 DEBUG:teuthology.parallel:result is None 2024-02-21T23:05:03.196 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-21T23:05:03.205 INFO:teuthology.task.internal:Creating test directory... 2024-02-21T23:05:03.205 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-21T23:05:03.207 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-21T23:05:03.226 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-21T23:05:03.242 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-21T23:05:03.276 INFO:teuthology.task.internal:Creating archive directory... 2024-02-21T23:05:03.276 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-21T23:05:03.279 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-21T23:05:03.310 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-21T23:05:03.317 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-21T23:05:03.318 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-21T23:05:03.338 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-21T23:05:03.374 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-21T23:05:03.384 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-21T23:05:03.387 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-21T23:05:03.398 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-21T23:05:03.400 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-21T23:05:03.407 INFO:teuthology.task.internal:Configuring sudo... 2024-02-21T23:05:03.407 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-21T23:05:03.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-21T23:05:03.471 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-21T23:05:03.484 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-21T23:05:03.484 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-21T23:05:03.498 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-21T23:05:03.527 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-21T23:05:03.594 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-21T23:05:03.650 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-21T23:05:03.732 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-21T23:05:03.773 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:05:03.773 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-21T23:05:03.844 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-21T23:05:03.886 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-21T23:05:03.946 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-21T23:05:04.026 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-21T23:05:04.072 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:05:04.073 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-21T23:05:04.140 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-02-21T23:05:04.142 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-02-21T23:05:04.187 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-21T23:05:04.230 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-21T23:05:04.704 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-21T23:05:04.715 INFO:teuthology.task.internal:Starting timer... 2024-02-21T23:05:04.715 INFO:teuthology.run_tasks:Running task pcp... 2024-02-21T23:05:04.765 INFO:teuthology.run_tasks:Running task selinux... 2024-02-21T23:05:04.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-02-21T23:05:04.847 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-02-21T23:05:04.848 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-02-21T23:05:04.911 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-02-21T23:05:04.912 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-21T23:05:04.912 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-02-21T23:05:04.930 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-02-21T23:05:04.930 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-02-21T23:05:04.952 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-02-21T23:05:04.952 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi133.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2024-02-21T23:05:04.952 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-21T23:05:04.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-21T23:05:04.999 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-21T23:05:05.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-21T23:05:05.026 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-21T23:05:05.026 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-02-21T23:05:05.116 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-02-21T23:05:05.154 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-21T23:05:05.164 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-21T23:05:05.181 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-21T23:05:05.182 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-21T23:10:03.625 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-02-21T23:10:03.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-02-21T23:10:03.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T23:10:03.724 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-02-21T23:10:03.799 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-02-21T23:10:03.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-21T23:10:03.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T23:10:03.892 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-02-21T23:10:03.988 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-21T23:10:03.989 INFO:teuthology.run_tasks:Running task clock... 2024-02-21T23:10:03.999 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-21T23:10:03.999 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-21T23:10:04.000 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-21T23:10:04.003 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-21T23:10:04.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-21T23:10:04.032 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-21T23:10:04.049 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-21T23:10:04.062 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-21T23:10:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-02-21T23:10:04.079 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-21T23:10:04.091 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-02-21T23:10:04.108 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-21T23:10:04.109 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-02-21T23:10:04.124 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-02-21T23:10:04.125 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-21T23:10:04.140 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-21T23:10:04.157 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-21T23:10:04.175 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-02-21T23:10:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-21T23:10:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-02-21T23:10:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-21T23:10:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-21T23:10:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-21T23:10:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-21T23:10:04.211 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-02-21T23:10:04.214 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-21T23:10:04.214 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-02-21T23:10:04.214 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-21T23:10:04.214 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-21T23:10:04.214 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-21T23:10:04.214 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-21T23:10:04.215 INFO:teuthology.run_tasks:Running task install... 2024-02-21T23:10:04.223 DEBUG:teuthology.task.install:project ceph 2024-02-21T23:10:04.223 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f'}} 2024-02-21T23:10:04.223 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'debuginfo': True, 'flavor': 'default', 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f'} 2024-02-21T23:10:04.223 INFO:teuthology.task.install:Using flavor: default 2024-02-21T23:10:04.230 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2024-02-21T23:10:04.230 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-21T23:10:04.230 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f', 'tag': None, 'wait_for_package': False} 2024-02-21T23:10:04.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3bfc837d7b45ce8e095d92fa9c342964cea5e95f 2024-02-21T23:10:04.233 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f', 'tag': None, 'wait_for_package': False} 2024-02-21T23:10:04.233 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3bfc837d7b45ce8e095d92fa9c342964cea5e95f 2024-02-21T23:10:04.449 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/centos/9/flavors/default/ 2024-02-21T23:10:04.450 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1643.g3bfc837d 2024-02-21T23:10:04.458 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/centos/9/flavors/default/ 2024-02-21T23:10:04.459 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1643.g3bfc837d 2024-02-21T23:10:04.559 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-21T23:10:04.559 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:10:04.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-21T23:10:04.568 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-21T23:10:04.568 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:10:04.568 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-21T23:10:04.587 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-21T23:10:04.588 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-21T23:10:04.596 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-21T23:10:04.597 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3bfc837d7b45ce8e095d92fa9c342964cea5e95f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-21T23:10:04.655 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-21T23:10:04.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-21T23:10:04.735 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-21T23:10:04.745 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-21T23:10:04.801 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-02-21T23:10:04.803 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-02-21T23:10:04.812 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-02-21T23:10:04.814 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-02-21T23:10:05.055 INFO:teuthology.orchestra.run.smithi133.stdout:74 files removed 2024-02-21T23:10:05.057 INFO:teuthology.orchestra.run.smithi142.stdout:74 files removed 2024-02-21T23:10:05.083 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable-object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph-base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs-debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw-debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-21T23:10:05.084 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable-object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph-base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs-debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw-debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-21T23:10:05.795 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 211 kB/s | 83 kB 00:00 2024-02-21T23:10:05.815 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-02-21T23:10:06.241 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-02-21T23:10:06.246 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-02-21T23:10:06.608 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-02-21T23:10:06.688 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-02-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - AppStream 19 MB/s | 8.1 MB 00:00 2024-02-21T23:10:07.332 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-02-21T23:10:08.728 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.5 MB 00:00 2024-02-21T23:10:08.958 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.5 MB 00:00 2024-02-21T23:10:09.710 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-02-21T23:10:11.031 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-02-21T23:10:28.359 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-02-21T23:10:33.659 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.1 MB 00:01 2024-02-21T23:10:35.260 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 15 kB 00:00 2024-02-21T23:10:37.413 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:01 2024-02-21T23:10:42.527 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 8.3 kB/s | 1.7 kB 00:00 2024-02-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi142.stderr:Modular dependency problems: 2024-02-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-02-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 1: conflicting requests 2024-02-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 2: conflicting requests 2024-02-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 3: conflicting requests 2024-02-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 4: conflicting requests 2024-02-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 5: conflicting requests 2024-02-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 6: conflicting requests 2024-02-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 7: conflicting requests 2024-02-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 8: conflicting requests 2024-02-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 9: conflicting requests 2024-02-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 10: conflicting requests 2024-02-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 11: conflicting requests 2024-02-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 12: conflicting requests 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 13: conflicting requests 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 14: conflicting requests 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 15: conflicting requests 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 16: conflicting requests 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 17: conflicting requests 2024-02-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 18: conflicting requests 2024-02-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 19: conflicting requests 2024-02-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 20: conflicting requests 2024-02-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 21: conflicting requests 2024-02-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 22: conflicting requests 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 23: conflicting requests 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 24: conflicting requests 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 25: conflicting requests 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 26: conflicting requests 2024-02-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 27: conflicting requests 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 28: conflicting requests 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 29: conflicting requests 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 30: conflicting requests 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 31: conflicting requests 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 32: conflicting requests 2024-02-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 33: conflicting requests 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 34: conflicting requests 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 35: conflicting requests 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 36: conflicting requests 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 37: cannot install the best candidate for the job 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 38: conflicting requests 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 39: cannot install the best candidate for the job 2024-02-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 40: cannot install the best candidate for the job 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 41: cannot install the best candidate for the job 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.088 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 42: cannot install the best candidate for the job 2024-02-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 43: cannot install the best candidate for the job 2024-02-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi142.stderr: Problem 44: cannot install the best candidate for the job 2024-02-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi142.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-21T23:10:45.531 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-21T23:10:45.532 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-21T23:10:45.562 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-21T23:10:45.563 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-21T23:10:45.563 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-21T23:10:45.564 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-21T23:10:45.566 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-21T23:10:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-21T23:10:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-21T23:10:45.659 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-02-21T23:10:45.683 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================================= 2024-02-21T23:10:45.683 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-02-21T23:10:45.683 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================================= 2024-02-21T23:10:45.683 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-02-21T23:10:45.683 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-21T23:10:45.684 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 6.6 k 2024-02-21T23:10:45.684 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 5.3 M 2024-02-21T23:10:45.684 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 68 M 2024-02-21T23:10:45.684 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 436 M 2024-02-21T23:10:45.684 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 889 k 2024-02-21T23:10:45.684 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 6.0 M 2024-02-21T23:10:45.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 145 k 2024-02-21T23:10:45.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 1.4 M 2024-02-21T23:10:45.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 52 M 2024-02-21T23:10:45.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 1.5 M 2024-02-21T23:10:45.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 138 k 2024-02-21T23:10:45.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 1.8 M 2024-02-21T23:10:45.686 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 20 M 2024-02-21T23:10:45.686 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 7.4 M 2024-02-21T23:10:45.686 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 49 k 2024-02-21T23:10:45.686 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 99 M 2024-02-21T23:10:45.686 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 288 M 2024-02-21T23:10:45.686 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 9.8 M 2024-02-21T23:10:45.687 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 176 M 2024-02-21T23:10:45.687 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 47 M 2024-02-21T23:10:45.687 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 675 M 2024-02-21T23:10:45.687 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 264 k 2024-02-21T23:10:45.687 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 748 k 2024-02-21T23:10:45.688 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 220 k 2024-02-21T23:10:45.688 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 27 k 2024-02-21T23:10:45.688 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 18 k 2024-02-21T23:10:45.688 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-21T23:10:45.688 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-21T23:10:45.689 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-21T23:10:45.689 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 31 k 2024-02-21T23:10:45.689 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 745 k 2024-02-21T23:10:45.689 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 5.4 M 2024-02-21T23:10:45.689 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 127 k 2024-02-21T23:10:45.689 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 109 k 2024-02-21T23:10:45.690 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 45 M 2024-02-21T23:10:45.690 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 52 M 2024-02-21T23:10:45.690 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-21T23:10:45.690 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-21T23:10:45.690 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-21T23:10:45.690 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-21T23:10:45.691 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-21T23:10:45.691 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-21T23:10:45.691 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 161 k 2024-02-21T23:10:45.691 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 615 k 2024-02-21T23:10:45.691 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 321 k 2024-02-21T23:10:45.691 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 770 k 2024-02-21T23:10:45.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 298 k 2024-02-21T23:10:45.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 1.1 M 2024-02-21T23:10:45.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 100 k 2024-02-21T23:10:45.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 337 k 2024-02-21T23:10:45.692 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 85 k 2024-02-21T23:10:45.692 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 516 k 2024-02-21T23:10:45.693 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 3.1 M 2024-02-21T23:10:45.693 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 52 M 2024-02-21T23:10:45.693 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 170 k 2024-02-21T23:10:45.693 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 1.3 M 2024-02-21T23:10:45.693 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-21T23:10:45.693 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-02-21T23:10:45.694 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-21T23:10:45.694 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-21T23:10:45.694 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-21T23:10:45.694 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-21T23:10:45.694 INFO:teuthology.orchestra.run.smithi142.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-21T23:10:45.694 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-21T23:10:45.695 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-21T23:10:45.695 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-21T23:10:45.695 INFO:teuthology.orchestra.run.smithi142.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-21T23:10:45.695 INFO:teuthology.orchestra.run.smithi142.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-21T23:10:45.695 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 3.4 M 2024-02-21T23:10:45.695 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 3.1 M 2024-02-21T23:10:45.696 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-21T23:10:45.696 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-21T23:10:45.696 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-21T23:10:45.696 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-21T23:10:45.696 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-02-21T23:10:45.697 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-21T23:10:45.697 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-21T23:10:45.697 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-21T23:10:45.698 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 20 M 2024-02-21T23:10:45.698 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 23 M 2024-02-21T23:10:45.698 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 31 k 2024-02-21T23:10:45.699 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 2.3 M 2024-02-21T23:10:45.699 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 248 k 2024-02-21T23:10:45.699 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 4.7 M 2024-02-21T23:10:45.699 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 17 M 2024-02-21T23:10:45.699 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 15 k 2024-02-21T23:10:45.699 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 25 k 2024-02-21T23:10:45.699 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-21T23:10:45.700 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-21T23:10:45.700 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-21T23:10:45.700 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-21T23:10:45.700 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-21T23:10:45.700 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-21T23:10:45.700 INFO:teuthology.orchestra.run.smithi142.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-21T23:10:45.700 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-21T23:10:45.700 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-21T23:10:45.701 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 163 k 2024-02-21T23:10:45.701 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-21T23:10:45.701 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-21T23:10:45.701 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-21T23:10:45.701 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-21T23:10:45.701 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-21T23:10:45.702 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 502 k 2024-02-21T23:10:45.702 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-21T23:10:45.702 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 5.1 M 2024-02-21T23:10:45.702 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-21T23:10:45.702 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-21T23:10:45.702 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-21T23:10:45.703 INFO:teuthology.orchestra.run.smithi142.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-21T23:10:45.703 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-21T23:10:45.703 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-21T23:10:45.703 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-21T23:10:45.703 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-21T23:10:45.703 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-21T23:10:45.703 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-21T23:10:45.704 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-21T23:10:45.704 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-21T23:10:45.704 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-21T23:10:45.704 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-21T23:10:45.704 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-21T23:10:45.704 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-21T23:10:45.704 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-21T23:10:45.704 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-21T23:10:45.705 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 45 k 2024-02-21T23:10:45.705 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 131 k 2024-02-21T23:10:45.705 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-21T23:10:45.705 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-21T23:10:45.705 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-21T23:10:45.706 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-21T23:10:45.706 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-21T23:10:45.706 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-21T23:10:45.706 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-21T23:10:45.709 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-21T23:10:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-21T23:10:45.711 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-21T23:10:45.711 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-21T23:10:45.711 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-21T23:10:45.711 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-21T23:10:45.711 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-21T23:10:45.711 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-21T23:10:45.711 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-21T23:10:45.711 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-21T23:10:45.711 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-21T23:10:45.712 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-21T23:10:45.713 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-21T23:10:45.714 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-21T23:10:45.714 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-02-21T23:10:45.714 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-21T23:10:45.714 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:10:45.714 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-02-21T23:10:45.714 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================================= 2024-02-21T23:10:45.714 INFO:teuthology.orchestra.run.smithi142.stdout:Install 172 Packages 2024-02-21T23:10:45.714 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 16 Packages 2024-02-21T23:10:45.714 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:10:45.715 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 2.2 G 2024-02-21T23:10:45.715 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-02-21T23:10:46.345 INFO:teuthology.orchestra.run.smithi142.stdout:(1/188): ceph-19.0.0-1643.g3bfc837d.el9.x86_64. 43 kB/s | 6.6 kB 00:00 2024-02-21T23:10:47.212 INFO:teuthology.orchestra.run.smithi142.stdout:(2/188): ceph-base-19.0.0-1643.g3bfc837d.el9.x8 5.2 MB/s | 5.3 MB 00:01 2024-02-21T23:10:52.097 INFO:teuthology.orchestra.run.smithi142.stdout:(3/188): ceph-base-debuginfo-19.0.0-1643.g3bfc8 12 MB/s | 68 MB 00:05 2024-02-21T23:10:53.720 INFO:teuthology.orchestra.run.smithi142.stdout:(4/188): ceph-common-19.0.0-1643.g3bfc837d.el9. 2.8 MB/s | 20 MB 00:07 2024-02-21T23:10:53.937 INFO:teuthology.orchestra.run.smithi142.stdout:(5/188): ceph-debuginfo-19.0.0-1643.g3bfc837d.e 12 MB/s | 23 MB 00:01 2024-02-21T23:10:53.995 INFO:teuthology.orchestra.run.smithi142.stdout:(6/188): ceph-fuse-19.0.0-1643.g3bfc837d.el9.x8 3.2 MB/s | 889 kB 00:00 2024-02-21T23:10:54.562 INFO:teuthology.orchestra.run.smithi142.stdout:(7/188): ceph-fuse-debuginfo-19.0.0-1643.g3bfc8 9.6 MB/s | 6.0 MB 00:00 2024-02-21T23:10:54.587 INFO:teuthology.orchestra.run.smithi142.stdout:(8/188): ceph-immutable-object-cache-19.0.0-164 245 kB/s | 145 kB 00:00 2024-02-21T23:10:54.721 INFO:teuthology.orchestra.run.smithi142.stdout:(9/188): ceph-immutable-object-cache-debuginfo- 9.1 MB/s | 1.4 MB 00:00 2024-02-21T23:10:55.346 INFO:teuthology.orchestra.run.smithi142.stdout:(10/188): ceph-mds-19.0.0-1643.g3bfc837d.el9.x8 3.0 MB/s | 2.3 MB 00:00 2024-02-21T23:10:56.180 INFO:teuthology.orchestra.run.smithi142.stdout:(11/188): ceph-mgr-19.0.0-1643.g3bfc837d.el9.x8 1.8 MB/s | 1.5 MB 00:00 2024-02-21T23:11:04.567 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 373 kB/s | 19 MB 00:52 2024-02-21T23:11:06.579 INFO:teuthology.orchestra.run.smithi142.stdout:(12/188): ceph-mds-debuginfo-19.0.0-1643.g3bfc8 4.4 MB/s | 52 MB 00:11 2024-02-21T23:11:07.071 INFO:teuthology.orchestra.run.smithi142.stdout:(13/188): ceph-mon-19.0.0-1643.g3bfc837d.el9.x8 9.5 MB/s | 4.7 MB 00:00 2024-02-21T23:11:08.742 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.1 MB 00:00 2024-02-21T23:11:09.921 INFO:teuthology.orchestra.run.smithi142.stdout:(14/188): ceph-mgr-debuginfo-19.0.0-1643.g3bfc8 1.5 MB/s | 20 MB 00:13 2024-02-21T23:11:10.252 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 15 kB 00:00 2024-02-21T23:11:23.195 INFO:teuthology.orchestra.run.smithi142.stdout:(15/188): ceph-mon-debuginfo-19.0.0-1643.g3bfc8 6.1 MB/s | 99 MB 00:16 2024-02-21T23:11:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:(16/188): ceph-osd-19.0.0-1643.g3bfc837d.el9.x8 1.2 MB/s | 17 MB 00:13 2024-02-21T23:11:25.745 INFO:teuthology.orchestra.run.smithi142.stdout:(17/188): ceph-radosgw-19.0.0-1643.g3bfc837d.el 4.3 MB/s | 9.8 MB 00:02 2024-02-21T23:11:52.208 INFO:teuthology.orchestra.run.smithi142.stdout:(18/188): ceph-osd-debuginfo-19.0.0-1643.g3bfc8 9.9 MB/s | 288 MB 00:29 2024-02-21T23:11:52.291 INFO:teuthology.orchestra.run.smithi142.stdout:(19/188): ceph-selinux-19.0.0-1643.g3bfc837d.el 303 kB/s | 25 kB 00:00 2024-02-21T23:11:55.696 INFO:teuthology.orchestra.run.smithi142.stdout:(20/188): ceph-test-19.0.0-1643.g3bfc837d.el9.x 14 MB/s | 47 MB 00:03 2024-02-21T23:12:07.623 INFO:teuthology.orchestra.run.smithi142.stdout:(21/188): ceph-common-debuginfo-19.0.0-1643.g3b 5.4 MB/s | 436 MB 01:20 2024-02-21T23:12:08.032 INFO:teuthology.orchestra.run.smithi142.stdout:(22/188): cephfs-mirror-19.0.0-1643.g3bfc837d.e 539 kB/s | 220 kB 00:00 2024-02-21T23:12:08.148 INFO:teuthology.orchestra.run.smithi142.stdout:(23/188): cephfs-shell-19.0.0-1643.g3bfc837d.el 232 kB/s | 27 kB 00:00 2024-02-21T23:12:08.249 INFO:teuthology.orchestra.run.smithi142.stdout:(24/188): libcephfs-devel-19.0.0-1643.g3bfc837d 312 kB/s | 31 kB 00:00 2024-02-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi142.stdout:(25/188): libcephfs2-19.0.0-1643.g3bfc837d.el9. 1.0 MB/s | 745 kB 00:00 2024-02-21T23:12:13.698 INFO:teuthology.orchestra.run.smithi142.stdout:(26/188): ceph-radosgw-debuginfo-19.0.0-1643.g3 3.7 MB/s | 176 MB 00:47 2024-02-21T23:12:13.765 INFO:teuthology.orchestra.run.smithi142.stdout:(27/188): libcephfs2-debuginfo-19.0.0-1643.g3bf 1.1 MB/s | 5.4 MB 00:04 2024-02-21T23:12:13.857 INFO:teuthology.orchestra.run.smithi142.stdout:(28/188): libcephsqlite-19.0.0-1643.g3bfc837d.e 1.0 MB/s | 163 kB 00:00 2024-02-21T23:12:13.949 INFO:teuthology.orchestra.run.smithi142.stdout:(29/188): librados-devel-debuginfo-19.0.0-1643. 1.2 MB/s | 109 kB 00:00 2024-02-21T23:12:14.033 INFO:teuthology.orchestra.run.smithi142.stdout:(30/188): librados-devel-19.0.0-1643.g3bfc837d. 474 kB/s | 127 kB 00:00 2024-02-21T23:12:14.475 INFO:teuthology.orchestra.run.smithi142.stdout:(31/188): libradosstriper1-19.0.0-1643.g3bfc837 1.1 MB/s | 502 kB 00:00 2024-02-21T23:12:29.198 INFO:teuthology.orchestra.run.smithi142.stdout:(32/188): librados2-debuginfo-19.0.0-1643.g3bfc 3.0 MB/s | 45 MB 00:15 2024-02-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi142.stdout:(33/188): librgw2-19.0.0-1643.g3bfc837d.el9.x86 2.2 MB/s | 5.1 MB 00:02 2024-02-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi142.stdout:(34/188): python3-ceph-argparse-19.0.0-1643.g3b 1.1 MB/s | 45 kB 00:00 2024-02-21T23:12:31.640 INFO:teuthology.orchestra.run.smithi142.stdout:(35/188): python3-ceph-common-19.0.0-1643.g3bfc 2.2 MB/s | 131 kB 00:00 2024-02-21T23:12:31.707 INFO:teuthology.orchestra.run.smithi142.stdout:(36/188): python3-cephfs-19.0.0-1643.g3bfc837d. 2.4 MB/s | 161 kB 00:00 2024-02-21T23:12:31.849 INFO:teuthology.orchestra.run.smithi142.stdout:(37/188): python3-cephfs-debuginfo-19.0.0-1643. 4.2 MB/s | 615 kB 00:00 2024-02-21T23:12:31.924 INFO:teuthology.orchestra.run.smithi142.stdout:(38/188): python3-rados-19.0.0-1643.g3bfc837d.e 4.2 MB/s | 321 kB 00:00 2024-02-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi142.stdout:(39/188): python3-rados-debuginfo-19.0.0-1643.g 4.7 MB/s | 770 kB 00:00 2024-02-21T23:12:32.158 INFO:teuthology.orchestra.run.smithi142.stdout:(40/188): python3-rbd-19.0.0-1643.g3bfc837d.el9 3.9 MB/s | 298 kB 00:00 2024-02-21T23:12:32.358 INFO:teuthology.orchestra.run.smithi142.stdout:(41/188): python3-rbd-debuginfo-19.0.0-1643.g3b 5.6 MB/s | 1.1 MB 00:00 2024-02-21T23:12:32.425 INFO:teuthology.orchestra.run.smithi142.stdout:(42/188): python3-rgw-19.0.0-1643.g3bfc837d.el9 1.5 MB/s | 100 kB 00:00 2024-02-21T23:12:32.542 INFO:teuthology.orchestra.run.smithi142.stdout:(43/188): python3-rgw-debuginfo-19.0.0-1643.g3b 2.8 MB/s | 337 kB 00:00 2024-02-21T23:12:32.601 INFO:teuthology.orchestra.run.smithi142.stdout:(44/188): rbd-fuse-19.0.0-1643.g3bfc837d.el9.x8 1.4 MB/s | 85 kB 00:00 2024-02-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi142.stdout:(45/188): rbd-fuse-debuginfo-19.0.0-1643.g3bfc8 3.4 MB/s | 516 kB 00:00 2024-02-21T23:12:33.118 INFO:teuthology.orchestra.run.smithi142.stdout:(46/188): rbd-mirror-19.0.0-1643.g3bfc837d.el9. 8.4 MB/s | 3.1 MB 00:00 2024-02-21T23:12:35.959 INFO:teuthology.orchestra.run.smithi142.stdout:(47/188): librbd1-debuginfo-19.0.0-1643.g3bfc83 2.4 MB/s | 52 MB 00:21 2024-02-21T23:12:36.102 INFO:teuthology.orchestra.run.smithi142.stdout:(48/188): rbd-nbd-19.0.0-1643.g3bfc837d.el9.x86 1.2 MB/s | 170 kB 00:00 2024-02-21T23:12:36.460 INFO:teuthology.orchestra.run.smithi142.stdout:(49/188): rbd-nbd-debuginfo-19.0.0-1643.g3bfc83 3.6 MB/s | 1.3 MB 00:00 2024-02-21T23:12:36.535 INFO:teuthology.orchestra.run.smithi142.stdout:(50/188): ceph-grafana-dashboards-19.0.0-1643.g 421 kB/s | 31 kB 00:00 2024-02-21T23:12:36.610 INFO:teuthology.orchestra.run.smithi142.stdout:(51/188): ceph-mgr-cephadm-19.0.0-1643.g3bfc837 1.8 MB/s | 138 kB 00:00 2024-02-21T23:12:37.444 INFO:teuthology.orchestra.run.smithi142.stdout:(52/188): ceph-mgr-dashboard-19.0.0-1643.g3bfc8 2.1 MB/s | 1.8 MB 00:00 2024-02-21T23:12:38.886 INFO:teuthology.orchestra.run.smithi142.stdout:(53/188): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-02-21T23:12:38.994 INFO:teuthology.orchestra.run.smithi142.stdout:(54/188): ceph-mgr-modules-core-19.0.0-1643.g3b 2.2 MB/s | 248 kB 00:00 2024-02-21T23:12:39.069 INFO:teuthology.orchestra.run.smithi142.stdout:(55/188): ceph-mgr-rook-19.0.0-1643.g3bfc837d.e 658 kB/s | 49 kB 00:00 2024-02-21T23:12:39.512 INFO:teuthology.orchestra.run.smithi142.stdout:(56/188): rbd-mirror-debuginfo-19.0.0-1643.g3bf 8.1 MB/s | 52 MB 00:06 2024-02-21T23:12:39.528 INFO:teuthology.orchestra.run.smithi142.stdout:(57/188): ceph-prometheus-alerts-19.0.0-1643.g3 33 kB/s | 15 kB 00:00 2024-02-21T23:12:39.604 INFO:teuthology.orchestra.run.smithi142.stdout:(58/188): ceph-volume-19.0.0-1643.g3bfc837d.el9 2.8 MB/s | 264 kB 00:00 2024-02-21T23:12:39.654 INFO:teuthology.orchestra.run.smithi142.stdout:(59/188): cephfs-top-19.0.0-1643.g3bfc837d.el9. 366 kB/s | 18 kB 00:00 2024-02-21T23:12:39.746 INFO:teuthology.orchestra.run.smithi142.stdout:(60/188): cephadm-19.0.0-1643.g3bfc837d.el9.noa 3.4 MB/s | 748 kB 00:00 2024-02-21T23:12:39.779 INFO:teuthology.orchestra.run.smithi142.stdout:(61/188): libaio-devel-0.3.112-1.el8.x86_64.rpm 141 kB/s | 18 kB 00:00 2024-02-21T23:12:39.863 INFO:teuthology.orchestra.run.smithi142.stdout:(62/188): fuse-2.9.9-15.el9.x86_64.rpm 696 kB/s | 80 kB 00:00 2024-02-21T23:12:39.922 INFO:teuthology.orchestra.run.smithi142.stdout:(63/188): jq-1.6-16.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-02-21T23:12:39.980 INFO:teuthology.orchestra.run.smithi142.stdout:(64/188): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-02-21T23:12:40.022 INFO:teuthology.orchestra.run.smithi142.stdout:(65/188): libgfortran-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2024-02-21T23:12:40.047 INFO:teuthology.orchestra.run.smithi142.stdout:(66/188): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 218 kB 00:00 2024-02-21T23:12:40.073 INFO:teuthology.orchestra.run.smithi142.stdout:(67/188): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-02-21T23:12:40.123 INFO:teuthology.orchestra.run.smithi142.stdout:(68/188): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-02-21T23:12:40.156 INFO:teuthology.orchestra.run.smithi142.stdout:(69/188): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-02-21T23:12:40.182 INFO:teuthology.orchestra.run.smithi142.stdout:(70/188): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-02-21T23:12:40.208 INFO:teuthology.orchestra.run.smithi142.stdout:(71/188): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-02-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi142.stdout:(72/188): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-02-21T23:12:40.558 INFO:teuthology.orchestra.run.smithi142.stdout:(73/188): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-02-21T23:12:40.625 INFO:teuthology.orchestra.run.smithi142.stdout:(74/188): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-02-21T23:12:40.675 INFO:teuthology.orchestra.run.smithi142.stdout:(75/188): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-02-21T23:12:40.725 INFO:teuthology.orchestra.run.smithi142.stdout:(76/188): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 313 kB 00:00 2024-02-21T23:12:40.775 INFO:teuthology.orchestra.run.smithi142.stdout:(77/188): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-02-21T23:12:40.809 INFO:teuthology.orchestra.run.smithi142.stdout:(78/188): bison-3.7.4-5.el9.x86_64.rpm 1.6 MB/s | 943 kB 00:00 2024-02-21T23:12:40.859 INFO:teuthology.orchestra.run.smithi142.stdout:(79/188): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-02-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi142.stdout:(80/188): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-02-21T23:12:41.018 INFO:teuthology.orchestra.run.smithi142.stdout:(81/188): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-02-21T23:12:41.044 INFO:teuthology.orchestra.run.smithi142.stdout:(82/188): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-02-21T23:12:41.096 INFO:teuthology.orchestra.run.smithi142.stdout:(83/188): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-02-21T23:12:41.144 INFO:teuthology.orchestra.run.smithi142.stdout:(84/188): libuuid-devel-2.37.4-15.el9.x86_64.rp 512 kB/s | 26 kB 00:00 2024-02-21T23:12:41.194 INFO:teuthology.orchestra.run.smithi142.stdout:(85/188): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-02-21T23:12:41.261 INFO:teuthology.orchestra.run.smithi142.stdout:(86/188): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-02-21T23:12:41.286 INFO:teuthology.orchestra.run.smithi142.stdout:(87/188): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-02-21T23:12:41.336 INFO:teuthology.orchestra.run.smithi142.stdout:(88/188): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-02-21T23:12:41.354 INFO:teuthology.orchestra.run.smithi142.stdout:(89/188): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-02-21T23:12:41.581 INFO:teuthology.orchestra.run.smithi142.stdout:(90/188): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-02-21T23:12:41.673 INFO:teuthology.orchestra.run.smithi142.stdout:(91/188): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-02-21T23:12:41.731 INFO:teuthology.orchestra.run.smithi142.stdout:(92/188): postgresql-contrib-13.11-1.el9.x86_64 15 MB/s | 905 kB 00:00 2024-02-21T23:12:41.782 INFO:teuthology.orchestra.run.smithi142.stdout:(93/188): postgresql-private-libs-13.11-1.el9.x 2.6 MB/s | 135 kB 00:00 2024-02-21T23:12:41.916 INFO:teuthology.orchestra.run.smithi142.stdout:(94/188): openssl-devel-3.0.7-25.el9.x86_64.rpm 7.3 MB/s | 4.1 MB 00:00 2024-02-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi142.stdout:(95/188): postgresql-server-13.11-1.el9.x86_64. 23 MB/s | 5.7 MB 00:00 2024-02-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi142.stdout:(96/188): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-02-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi142.stdout:(97/188): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-02-21T23:12:42.150 INFO:teuthology.orchestra.run.smithi142.stdout:(98/188): python3-devel-3.9.17-2.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-02-21T23:12:42.200 INFO:teuthology.orchestra.run.smithi142.stdout:(99/188): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-02-21T23:12:42.250 INFO:teuthology.orchestra.run.smithi142.stdout:(100/188): python3-jsonpatch-1.21-16.el9.noarch 512 kB/s | 26 kB 00:00 2024-02-21T23:12:42.300 INFO:teuthology.orchestra.run.smithi142.stdout:(101/188): python3-jsonpointer-2.0-4.el9.noarch 383 kB/s | 19 kB 00:00 2024-02-21T23:12:42.351 INFO:teuthology.orchestra.run.smithi142.stdout:(102/188): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-02-21T23:12:42.401 INFO:teuthology.orchestra.run.smithi142.stdout:(103/188): python3-markupsafe-1.1.1-12.el9.x86_ 695 kB/s | 35 kB 00:00 2024-02-21T23:12:42.560 INFO:teuthology.orchestra.run.smithi142.stdout:(104/188): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-02-21T23:12:42.627 INFO:teuthology.orchestra.run.smithi142.stdout:(105/188): python3-numpy-1.20.1-5.el9.x86_64.rp 23 MB/s | 5.1 MB 00:00 2024-02-21T23:12:42.652 INFO:teuthology.orchestra.run.smithi142.stdout:(106/188): python3-numpy-f2py-1.20.1-5.el9.x86_ 2.7 MB/s | 250 kB 00:00 2024-02-21T23:12:42.677 INFO:teuthology.orchestra.run.smithi142.stdout:(107/188): python3-oauthlib-3.1.1-5.el9.noarch. 4.3 MB/s | 222 kB 00:00 2024-02-21T23:12:42.703 INFO:teuthology.orchestra.run.smithi142.stdout:(108/188): python3-packaging-20.9-5.el9.noarch. 1.5 MB/s | 77 kB 00:00 2024-02-21T23:12:42.728 INFO:teuthology.orchestra.run.smithi142.stdout:(109/188): python3-prettytable-0.7.2-27.el9.noa 844 kB/s | 42 kB 00:00 2024-02-21T23:12:42.753 INFO:teuthology.orchestra.run.smithi142.stdout:(110/188): python3-protobuf-3.14.0-13.el9.noarc 5.2 MB/s | 269 kB 00:00 2024-02-21T23:12:42.778 INFO:teuthology.orchestra.run.smithi142.stdout:(111/188): python3-pyasn1-0.4.8-6.el9.noarch.rp 3.1 MB/s | 159 kB 00:00 2024-02-21T23:12:42.803 INFO:teuthology.orchestra.run.smithi142.stdout:(112/188): python3-pyasn1-modules-0.4.8-6.el9.n 5.4 MB/s | 279 kB 00:00 2024-02-21T23:12:42.820 INFO:teuthology.orchestra.run.smithi142.stdout:(113/188): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-02-21T23:12:42.854 INFO:teuthology.orchestra.run.smithi142.stdout:(114/188): python3-requests-oauthlib-1.3.0-12.e 1.0 MB/s | 54 kB 00:00 2024-02-21T23:12:42.905 INFO:teuthology.orchestra.run.smithi142.stdout:(115/188): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-02-21T23:12:42.946 INFO:teuthology.orchestra.run.smithi142.stdout:(116/188): python3-wcwidth-0.2.5-8.el9.noarch.r 1.0 MB/s | 44 kB 00:00 2024-02-21T23:12:43.005 INFO:teuthology.orchestra.run.smithi142.stdout:(117/188): qatlib-23.11.0-1.el9.x86_64.rpm 3.6 MB/s | 216 kB 00:00 2024-02-21T23:12:43.047 INFO:teuthology.orchestra.run.smithi142.stdout:(118/188): qatlib-service-23.11.0-1.el9.x86_64. 838 kB/s | 35 kB 00:00 2024-02-21T23:12:43.089 INFO:teuthology.orchestra.run.smithi142.stdout:(119/188): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-21T23:12:43.156 INFO:teuthology.orchestra.run.smithi142.stdout:(120/188): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-02-21T23:12:43.198 INFO:teuthology.orchestra.run.smithi142.stdout:(121/188): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-02-21T23:12:43.349 INFO:teuthology.orchestra.run.smithi142.stdout:(122/188): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-02-21T23:12:43.374 INFO:teuthology.orchestra.run.smithi142.stdout:(123/188): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 534 kB/s | 94 kB 00:00 2024-02-21T23:12:43.391 INFO:teuthology.orchestra.run.smithi142.stdout:(124/188): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-02-21T23:12:43.416 INFO:teuthology.orchestra.run.smithi142.stdout:(125/188): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-02-21T23:12:43.551 INFO:teuthology.orchestra.run.smithi142.stdout:(126/188): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-02-21T23:12:43.760 INFO:teuthology.orchestra.run.smithi142.stdout:(127/188): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-02-21T23:12:44.208 INFO:teuthology.orchestra.run.smithi142.stdout:(128/188): abseil-cpp-20211102.0-3.el9.x86_64.r 832 kB/s | 548 kB 00:00 2024-02-21T23:12:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:(129/188): gperftools-libs-2.9.1-2.el9.x86_64.r 588 kB/s | 309 kB 00:00 2024-02-21T23:12:44.325 INFO:teuthology.orchestra.run.smithi142.stdout:(130/188): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-02-21T23:12:44.467 INFO:teuthology.orchestra.run.smithi142.stdout:(131/188): libarrow-doc-9.0.0-10.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-02-21T23:12:44.593 INFO:teuthology.orchestra.run.smithi142.stdout:(132/188): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-02-21T23:12:44.709 INFO:teuthology.orchestra.run.smithi142.stdout:(133/188): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-02-21T23:12:44.835 INFO:teuthology.orchestra.run.smithi142.stdout:(134/188): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-02-21T23:12:45.085 INFO:teuthology.orchestra.run.smithi142.stdout:(135/188): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-02-21T23:12:45.260 INFO:teuthology.orchestra.run.smithi142.stdout:(136/188): python3-asyncssh-2.13.2-4.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-02-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi142.stdout:(137/188): python3-bcrypt-3.2.2-1.el9.x86_64.rp 326 kB/s | 43 kB 00:00 2024-02-21T23:12:45.494 INFO:teuthology.orchestra.run.smithi142.stdout:(138/188): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-02-21T23:12:45.586 INFO:teuthology.orchestra.run.smithi142.stdout:(139/188): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-02-21T23:12:45.720 INFO:teuthology.orchestra.run.smithi142.stdout:(140/188): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-02-21T23:12:45.853 INFO:teuthology.orchestra.run.smithi142.stdout:(141/188): libarrow-9.0.0-10.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-02-21T23:12:45.887 INFO:teuthology.orchestra.run.smithi142.stdout:(142/188): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-02-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:(143/188): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-02-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:(144/188): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-02-21T23:12:46.096 INFO:teuthology.orchestra.run.smithi142.stdout:(145/188): python3-google-auth-2.27.0-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-02-21T23:12:46.215 INFO:teuthology.orchestra.run.smithi142.stdout:(146/188): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-02-21T23:12:46.372 INFO:teuthology.orchestra.run.smithi142.stdout:(147/188): python3-jaraco-8.2.1-3.el9.noarch.rp 68 kB/s | 11 kB 00:00 2024-02-21T23:12:46.480 INFO:teuthology.orchestra.run.smithi142.stdout:(148/188): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-02-21T23:12:46.531 INFO:teuthology.orchestra.run.smithi142.stdout:(149/188): python3-grpcio-1.46.7-10.el9.x86_64. 3.9 MB/s | 2.0 MB 00:00 2024-02-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi142.stdout:(150/188): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-02-21T23:12:46.631 INFO:teuthology.orchestra.run.smithi142.stdout:(151/188): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-02-21T23:12:46.682 INFO:teuthology.orchestra.run.smithi142.stdout:(152/188): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-02-21T23:12:46.790 INFO:teuthology.orchestra.run.smithi142.stdout:(153/188): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-02-21T23:12:46.899 INFO:teuthology.orchestra.run.smithi142.stdout:(154/188): python3-more-itertools-8.12.0-2.el9. 729 kB/s | 79 kB 00:00 2024-02-21T23:12:46.932 INFO:teuthology.orchestra.run.smithi142.stdout:(155/188): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-02-21T23:12:47.008 INFO:teuthology.orchestra.run.smithi142.stdout:(156/188): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-02-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi142.stdout:(157/188): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-02-21T23:12:47.158 INFO:teuthology.orchestra.run.smithi142.stdout:(158/188): python3-portend-3.1.0-2.el9.noarch.r 109 kB/s | 16 kB 00:00 2024-02-21T23:12:47.234 INFO:teuthology.orchestra.run.smithi142.stdout:(159/188): python3-pyOpenSSL-21.0.0-1.el9.noarc 898 kB/s | 90 kB 00:00 2024-02-21T23:12:47.267 INFO:teuthology.orchestra.run.smithi142.stdout:(160/188): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-02-21T23:12:47.334 INFO:teuthology.orchestra.run.smithi142.stdout:(161/188): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-02-21T23:12:47.401 INFO:teuthology.orchestra.run.smithi142.stdout:(162/188): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-02-21T23:12:47.435 INFO:teuthology.orchestra.run.smithi142.stdout:(163/188): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-02-21T23:12:47.502 INFO:teuthology.orchestra.run.smithi142.stdout:(164/188): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-02-21T23:12:47.535 INFO:teuthology.orchestra.run.smithi142.stdout:(165/188): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-02-21T23:12:47.635 INFO:teuthology.orchestra.run.smithi142.stdout:(166/188): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-02-21T23:12:47.661 INFO:teuthology.orchestra.run.smithi142.stdout:(167/188): python3-websocket-client-1.2.3-2.el9 716 kB/s | 90 kB 00:00 2024-02-21T23:12:47.761 INFO:teuthology.orchestra.run.smithi142.stdout:(168/188): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-02-21T23:12:47.811 INFO:teuthology.orchestra.run.smithi142.stdout:(169/188): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-02-21T23:12:47.861 INFO:teuthology.orchestra.run.smithi142.stdout:(170/188): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-21T23:12:48.103 INFO:teuthology.orchestra.run.smithi142.stdout:(171/188): thrift-0.15.0-2.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-02-21T23:12:48.479 INFO:teuthology.orchestra.run.smithi142.stdout:(172/188): librados2-19.0.0-1643.g3bfc837d.el9. 5.5 MB/s | 3.4 MB 00:00 2024-02-21T23:12:48.662 INFO:teuthology.orchestra.run.smithi142.stdout:(173/188): NetworkManager-1.45.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-02-21T23:12:48.737 INFO:teuthology.orchestra.run.smithi142.stdout:(174/188): NetworkManager-libnm-1.45.91-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-02-21T23:12:48.780 INFO:teuthology.orchestra.run.smithi142.stdout:(175/188): NetworkManager-team-1.45.91-1.el9.x8 1.0 MB/s | 42 kB 00:00 2024-02-21T23:12:48.821 INFO:teuthology.orchestra.run.smithi142.stdout:(176/188): NetworkManager-tui-1.45.91-1.el9.x86 5.8 MB/s | 247 kB 00:00 2024-02-21T23:12:48.863 INFO:teuthology.orchestra.run.smithi142.stdout:(177/188): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-02-21T23:12:48.913 INFO:teuthology.orchestra.run.smithi142.stdout:(178/188): librbd1-19.0.0-1643.g3bfc837d.el9.x8 3.9 MB/s | 3.1 MB 00:00 2024-02-21T23:12:48.931 INFO:teuthology.orchestra.run.smithi142.stdout:(179/188): elfutils-debuginfod-client-0.190-2.e 542 kB/s | 36 kB 00:00 2024-02-21T23:12:48.973 INFO:teuthology.orchestra.run.smithi142.stdout:(180/188): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-02-21T23:12:49.015 INFO:teuthology.orchestra.run.smithi142.stdout:(181/188): libacl-2.3.1-4.el9.x86_64.rpm 554 kB/s | 23 kB 00:00 2024-02-21T23:12:49.040 INFO:teuthology.orchestra.run.smithi142.stdout:(182/188): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 192 kB 00:00 2024-02-21T23:12:49.084 INFO:teuthology.orchestra.run.smithi142.stdout:(183/188): openssl-3.0.7-25.el9.x86_64.rpm 17 MB/s | 1.2 MB 00:00 2024-02-21T23:12:49.135 INFO:teuthology.orchestra.run.smithi142.stdout:(184/188): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2024-02-21T23:12:49.185 INFO:teuthology.orchestra.run.smithi142.stdout:(185/188): openssl-libs-3.0.7-25.el9.x86_64.rpm 15 MB/s | 2.2 MB 00:00 2024-02-21T23:12:49.227 INFO:teuthology.orchestra.run.smithi142.stdout:(186/188): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-02-21T23:12:49.419 INFO:teuthology.orchestra.run.smithi142.stdout:(187/188): libacl-devel-2.3.1-4.el9.x86_64.rpm 318 kB/s | 74 kB 00:00 2024-02-21T23:13:01.510 INFO:teuthology.orchestra.run.smithi142.stdout:(188/188): ceph-test-debuginfo-19.0.0-1643.g3bf 10 MB/s | 675 MB 01:05 2024-02-21T23:13:01.514 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-02-21T23:13:01.514 INFO:teuthology.orchestra.run.smithi142.stdout:Total 16 MB/s | 2.2 GB 02:15 2024-02-21T23:13:02.701 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-02-21T23:13:03.004 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-02-21T23:13:03.005 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-02-21T23:13:16.906 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-02-21T23:13:16.906 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-02-21T23:13:35.994 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-02-21T23:13:38.541 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/204 2024-02-21T23:13:38.668 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_6 2/204 2024-02-21T23:13:38.711 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/204 2024-02-21T23:13:38.769 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/204 2024-02-21T23:13:38.809 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/204 2024-02-21T23:13:38.843 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 6/204 2024-02-21T23:13:38.893 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 7/204 2024-02-21T23:13:39.224 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 8/204 2024-02-21T23:13:39.269 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:19.0.0-1643.g3bfc837d.el9.x86_64 9/204 2024-02-21T23:13:39.382 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:19.0.0-1643.g3bfc837d.el9.x86_64 9/204 2024-02-21T23:13:39.412 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:19.0.0-1643.g3bfc837d.el9.x86_64 10/204 2024-02-21T23:13:39.511 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:19.0.0-1643.g3bfc837d.el9.x86_64 10/204 2024-02-21T23:13:39.548 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:19.0.0-1643.g3bfc837d.el9.x86_64 11/204 2024-02-21T23:13:39.628 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/204 2024-02-21T23:13:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 13/204 2024-02-21T23:13:39.893 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/204 2024-02-21T23:13:39.950 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/204 2024-02-21T23:13:39.977 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:19.0.0-1643.g3bfc837d.el9.x86_64 16/204 2024-02-21T23:13:40.082 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1643.g3bfc837d.el9.x86_64 16/204 2024-02-21T23:13:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:19.0.0-1643.g3bfc837d.el9.x86 17/204 2024-02-21T23:13:40.149 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1643.g3bfc837d.el9.x86 17/204 2024-02-21T23:13:40.171 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 18/204 2024-02-21T23:13:40.186 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 18/204 2024-02-21T23:13:40.371 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 19/204 2024-02-21T23:13:40.400 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 19/204 2024-02-21T23:13:40.561 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/204 2024-02-21T23:13:40.702 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/204 2024-02-21T23:13:40.762 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/204 2024-02-21T23:13:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/204 2024-02-21T23:13:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 22/204 2024-02-21T23:13:41.127 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/204 2024-02-21T23:13:41.235 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-3.el9.x86_64 23/204 2024-02-21T23:13:41.330 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 24/204 2024-02-21T23:13:41.366 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 25/204 2024-02-21T23:13:41.403 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 26/204 2024-02-21T23:13:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 27/204 2024-02-21T23:13:41.505 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 28/204 2024-02-21T23:13:41.702 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 29/204 2024-02-21T23:13:41.887 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 30/204 2024-02-21T23:13:41.939 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-5.4.4-4.el9.x86_64 31/204 2024-02-21T23:13:42.009 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/204 2024-02-21T23:13:42.062 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 33/204 2024-02-21T23:13:42.093 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 34/204 2024-02-21T23:13:42.137 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 35/204 2024-02-21T23:13:42.213 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 36/204 2024-02-21T23:13:42.247 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:19.0.0-1643.g3bfc837d.el9. 37/204 2024-02-21T23:13:42.290 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:19.0.0-1643.g3bfc837d.el 38/204 2024-02-21T23:13:42.319 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:19.0.0-1643.g3bfc837d.el9.x86_6 39/204 2024-02-21T23:13:42.389 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 40/204 2024-02-21T23:13:42.433 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : luarocks-3.9.2-1.el9.noarch 41/204 2024-02-21T23:13:42.507 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 42/204 2024-02-21T23:13:42.660 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 43/204 2024-02-21T23:13:42.719 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 44/204 2024-02-21T23:13:43.259 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 45/204 2024-02-21T23:13:43.364 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 46/204 2024-02-21T23:13:43.392 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 47/204 2024-02-21T23:13:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 48/204 2024-02-21T23:13:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 49/204 2024-02-21T23:13:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:19.0.0-1643.g3bfc837d.el9.x86_6 50/204 2024-02-21T23:13:43.558 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 51/204 2024-02-21T23:13:43.699 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 52/204 2024-02-21T23:13:43.744 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 53/204 2024-02-21T23:13:43.816 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 54/204 2024-02-21T23:13:43.888 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 55/204 2024-02-21T23:13:43.928 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 56/204 2024-02-21T23:13:43.964 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 57/204 2024-02-21T23:13:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 58/204 2024-02-21T23:13:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 59/204 2024-02-21T23:13:44.090 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 60/204 2024-02-21T23:13:44.132 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 61/204 2024-02-21T23:13:44.266 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 62/204 2024-02-21T23:13:44.313 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 63/204 2024-02-21T23:13:44.343 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 64/204 2024-02-21T23:13:44.385 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 65/204 2024-02-21T23:13:44.478 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 66/204 2024-02-21T23:13:44.518 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 67/204 2024-02-21T23:13:44.559 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 68/204 2024-02-21T23:13:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 69/204 2024-02-21T23:13:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 70/204 2024-02-21T23:13:44.687 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 71/204 2024-02-21T23:13:44.817 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 72/204 2024-02-21T23:13:44.857 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 73/204 2024-02-21T23:13:44.885 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 74/204 2024-02-21T23:13:44.922 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 75/204 2024-02-21T23:13:44.958 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-55.el9.x86_64 76/204 2024-02-21T23:13:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 77/204 2024-02-21T23:13:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 78/204 2024-02-21T23:13:45.062 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 79/204 2024-02-21T23:13:45.096 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 80/204 2024-02-21T23:13:45.137 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 81/204 2024-02-21T23:13:45.171 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 82/204 2024-02-21T23:13:45.258 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 83/204 2024-02-21T23:13:45.310 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 84/204 2024-02-21T23:13:45.341 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 85/204 2024-02-21T23:13:45.389 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 86/204 2024-02-21T23:13:45.486 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 87/204 2024-02-21T23:13:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 88/204 2024-02-21T23:13:46.261 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 89/204 2024-02-21T23:13:46.307 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 90/204 2024-02-21T23:13:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 91/204 2024-02-21T23:13:46.438 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 92/204 2024-02-21T23:13:46.498 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 93/204 2024-02-21T23:13:46.593 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 94/204 2024-02-21T23:13:46.636 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 95/204 2024-02-21T23:13:46.813 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/204 2024-02-21T23:13:46.842 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/204 2024-02-21T23:13:46.923 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/204 2024-02-21T23:13:47.570 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/204 2024-02-21T23:13:47.689 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/204 2024-02-21T23:13:48.951 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/204 2024-02-21T23:13:49.037 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/204 2024-02-21T23:13:49.065 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 103/204 2024-02-21T23:13:49.098 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 104/204 2024-02-21T23:13:49.418 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/204 2024-02-21T23:13:49.453 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:19.0.0-1643.g3bfc837d.el9.x86_64 106/204 2024-02-21T23:13:49.535 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:19.0.0-1643.g3bfc837d.el9.x86_64 106/204 2024-02-21T23:13:49.566 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 107/204 2024-02-21T23:13:49.603 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/204 2024-02-21T23:13:50.056 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 109/204 2024-02-21T23:13:50.089 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:19.0.0-1643.g3bfc837d.el9.x86_64 110/204 2024-02-21T23:13:50.160 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:19.0.0-1643.g3bfc837d.el9.x86_64 110/204 2024-02-21T23:13:50.191 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 111/204 2024-02-21T23:13:52.453 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 112/204 2024-02-21T23:13:52.495 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 112/204 2024-02-21T23:13:53.044 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 112/204 2024-02-21T23:13:53.090 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:19.0.0-1643.g3bfc837d.el9.x86_64 113/204 2024-02-21T23:13:53.177 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:19.0.0-1643.g3bfc837d.el9.x86_64 113/204 2024-02-21T23:13:53.177 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-21T23:13:53.177 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-21T23:13:53.177 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:13:53.217 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:19.0.0-1643.g3bfc837d.el9.x86_64 114/204 2024-02-21T23:14:05.439 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1643.g3bfc837d.el9.x86_64 114/204 2024-02-21T23:14:05.439 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-02-21T23:14:05.439 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-02-21T23:14:05.439 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-02-21T23:14:05.439 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-02-21T23:14:05.439 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-02-21T23:14:05.439 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-02-21T23:14:05.439 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-02-21T23:14:05.439 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:05.602 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:19.0.0-1643.g3bfc837d.el9.noarch 115/204 2024-02-21T23:14:05.617 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:19.0.0-1643.g3bfc837d.el9.noarch 115/204 2024-02-21T23:14:05.617 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:14:05.617 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-21T23:14:05.617 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:19.0.0-1643.g3bfc837d.el9.x86_64 116/204 2024-02-21T23:14:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:19.0.0-1643.g3bfc837d.el9.x86_64 116/204 2024-02-21T23:14:07.138 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:14:07.138 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-21T23:14:07.138 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-21T23:14:07.139 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-21T23:14:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:07.364 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:19.0.0-1643.g3bfc837d.el9.x86_64 117/204 2024-02-21T23:14:07.386 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:19.0.0-1643.g3bfc837d.el9.x86_64 117/204 2024-02-21T23:14:07.386 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:14:07.386 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-21T23:14:07.386 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-21T23:14:07.386 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-21T23:14:07.386 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:07.840 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:19.0.0-1643.g3bfc837d.el9.x86_64 118/204 2024-02-21T23:14:07.862 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:19.0.0-1643.g3bfc837d.el9.x86_64 118/204 2024-02-21T23:14:07.862 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:14:07.862 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-21T23:14:07.862 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-21T23:14:07.862 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-21T23:14:07.862 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:07.970 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 119/204 2024-02-21T23:14:08.027 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 120/204 2024-02-21T23:14:08.082 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 121/204 2024-02-21T23:14:08.274 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 122/204 2024-02-21T23:14:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 123/204 2024-02-21T23:14:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/204 2024-02-21T23:14:08.498 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 125/204 2024-02-21T23:14:08.636 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 126/204 2024-02-21T23:14:08.674 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 127/204 2024-02-21T23:14:08.707 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 128/204 2024-02-21T23:14:08.739 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 129/204 2024-02-21T23:14:08.773 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : jq-1.6-16.el9.x86_64 130/204 2024-02-21T23:14:08.802 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-15.el9.x86_64 131/204 2024-02-21T23:14:09.211 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 132/204 2024-02-21T23:14:09.241 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 132/204 2024-02-21T23:14:09.273 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1643.g3bfc837d.e 133/204 2024-02-21T23:14:09.333 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1643.g3bfc837d. 134/204 2024-02-21T23:14:09.356 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1643.g3bfc837d.el9.noa 135/204 2024-02-21T23:14:09.584 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1643.g3bfc837d.el9.noa 135/204 2024-02-21T23:14:09.616 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1643.g3bfc837d.el9.n 136/204 2024-02-21T23:14:10.582 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1643.g3bfc837d.el9.n 136/204 2024-02-21T23:14:10.624 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1643.g3bf 137/204 2024-02-21T23:14:10.725 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1643.g3bf 137/204 2024-02-21T23:14:10.886 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1643.g3bfc837d.el 138/204 2024-02-21T23:14:10.927 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:19.0.0-1643.g3bfc837d.el9.x86_64 139/204 2024-02-21T23:14:10.949 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1643.g3bfc837d.el9.x86_64 139/204 2024-02-21T23:14:10.949 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:14:10.949 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-21T23:14:10.949 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-21T23:14:10.949 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-21T23:14:10.950 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:10.994 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:19.0.0-1643.g3bfc837d.el9.noarch 140/204 2024-02-21T23:14:11.004 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1643.g3bfc837d.el9.noarch 140/204 2024-02-21T23:14:11.103 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:19.0.0-1643.g3bfc837d.el9.x86_64 141/204 2024-02-21T23:14:15.536 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 142/204 2024-02-21T23:14:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:19.0.0-1643.g3bfc837d.el9.x86_64 143/204 2024-02-21T23:14:16.456 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-45.el9.x86_64 144/204 2024-02-21T23:14:16.492 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 145/204 2024-02-21T23:14:16.514 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 145/204 2024-02-21T23:14:16.515 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:14:16.515 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-21T23:14:16.515 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-21T23:14:16.515 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-21T23:14:16.515 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1643.g3bfc8 146/204 2024-02-21T23:14:16.581 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1643.g3bfc8 146/204 2024-02-21T23:14:16.581 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:14:16.581 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-21T23:14:16.581 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 147/204 2024-02-21T23:14:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 147/204 2024-02-21T23:14:16.649 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:14:16.649 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-21T23:14:16.649 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-21T23:14:16.649 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-21T23:14:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:16.952 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 148/204 2024-02-21T23:14:16.975 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 148/204 2024-02-21T23:14:16.975 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:14:16.975 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-21T23:14:16.975 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-21T23:14:16.975 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-21T23:14:16.975 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:14:17.027 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 149/204 2024-02-21T23:14:17.068 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 150/204 2024-02-21T23:14:17.100 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 151/204 2024-02-21T23:14:17.131 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 152/204 2024-02-21T23:14:17.307 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-shell-2:19.0.0-1643.g3bfc837d.el9.x86_64 153/204 2024-02-21T23:14:17.349 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 154/204 2024-02-21T23:14:17.382 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:19.0.0-1643.g3bfc837d.el9.x86_ 155/204 2024-02-21T23:14:17.430 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 156/204 2024-02-21T23:14:17.460 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 157/204 2024-02-21T23:14:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 158/204 2024-02-21T23:14:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 159/204 2024-02-21T23:14:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 160/204 2024-02-21T23:14:18.253 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 161/204 2024-02-21T23:14:18.290 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 161/204 2024-02-21T23:14:18.337 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 161/204 2024-02-21T23:14:18.387 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/204 2024-02-21T23:14:25.482 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:19.0.0-1643.g3bfc837d.el9.noarch 163/204 2024-02-21T23:15:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 164/204 2024-02-21T23:15:07.810 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-debuginfo-2:19.0.0-1643.g3bfc837d.el 165/204 2024-02-21T23:15:07.988 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 166/204 2024-02-21T23:15:12.797 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.0.0-1 167/204 2024-02-21T23:15:15.118 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 168/204 2024-02-21T23:15:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 169/204 2024-02-21T23:15:32.720 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 80 kB/s | 20 MB 04:22 2024-02-21T23:15:37.845 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr:Modular dependency problems: 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 1: conflicting requests 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 2: conflicting requests 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 3: conflicting requests 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 4: conflicting requests 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 5: conflicting requests 2024-02-21T23:15:40.399 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 6: conflicting requests 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 7: conflicting requests 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 8: conflicting requests 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 9: conflicting requests 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 10: conflicting requests 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 11: conflicting requests 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.400 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 12: conflicting requests 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 13: conflicting requests 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 14: conflicting requests 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 15: conflicting requests 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 16: conflicting requests 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 17: conflicting requests 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.401 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 18: conflicting requests 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 19: conflicting requests 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 20: conflicting requests 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 21: conflicting requests 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 22: conflicting requests 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 23: conflicting requests 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.402 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 24: conflicting requests 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 25: conflicting requests 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 26: conflicting requests 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 27: conflicting requests 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 28: conflicting requests 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 29: conflicting requests 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 30: conflicting requests 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.403 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 31: conflicting requests 2024-02-21T23:15:40.404 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.404 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 32: conflicting requests 2024-02-21T23:15:40.404 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.404 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 33: conflicting requests 2024-02-21T23:15:40.404 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.404 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 34: conflicting requests 2024-02-21T23:15:40.404 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.404 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 35: conflicting requests 2024-02-21T23:15:40.404 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 36: conflicting requests 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 37: cannot install the best candidate for the job 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 38: conflicting requests 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.405 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.406 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.406 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.406 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 39: cannot install the best candidate for the job 2024-02-21T23:15:40.406 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.406 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.406 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 40: cannot install the best candidate for the job 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 41: cannot install the best candidate for the job 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 42: cannot install the best candidate for the job 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.407 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 43: cannot install the best candidate for the job 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 44: cannot install the best candidate for the job 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.408 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-21T23:15:40.858 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-21T23:15:40.859 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-21T23:15:40.890 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-21T23:15:40.891 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-21T23:15:40.892 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-21T23:15:40.893 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-21T23:15:40.895 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-21T23:15:40.898 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-21T23:15:40.899 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-21T23:15:40.990 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-02-21T23:15:41.012 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================================= 2024-02-21T23:15:41.012 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-02-21T23:15:41.012 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================================= 2024-02-21T23:15:41.012 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-02-21T23:15:41.012 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-21T23:15:41.012 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 6.6 k 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 5.3 M 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 68 M 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 436 M 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 889 k 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 6.0 M 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 145 k 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 1.4 M 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 52 M 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 1.5 M 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 138 k 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 1.8 M 2024-02-21T23:15:41.013 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 20 M 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 7.4 M 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 49 k 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 99 M 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 288 M 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 9.8 M 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 176 M 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 47 M 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 675 M 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 264 k 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 748 k 2024-02-21T23:15:41.014 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 220 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 27 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 18 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 31 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 745 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 5.4 M 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 127 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 109 k 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 45 M 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 52 M 2024-02-21T23:15:41.015 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-21T23:15:41.016 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-21T23:15:41.016 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-21T23:15:41.016 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-21T23:15:41.016 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-21T23:15:41.016 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-21T23:15:41.016 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 161 k 2024-02-21T23:15:41.016 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 615 k 2024-02-21T23:15:41.016 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 321 k 2024-02-21T23:15:41.016 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 770 k 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 298 k 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 1.1 M 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 100 k 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 337 k 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 85 k 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 516 k 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 3.1 M 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 52 M 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 170 k 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 1.3 M 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-21T23:15:41.017 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-21T23:15:41.018 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 3.4 M 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 3.1 M 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 20 M 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-debuginfo x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 23 M 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 31 k 2024-02-21T23:15:41.019 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 2.3 M 2024-02-21T23:15:41.020 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 248 k 2024-02-21T23:15:41.020 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 4.7 M 2024-02-21T23:15:41.020 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 17 M 2024-02-21T23:15:41.020 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1643.g3bfc837d.el9 ceph-noarch 15 k 2024-02-21T23:15:41.020 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 25 k 2024-02-21T23:15:41.020 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-21T23:15:41.020 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 163 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-21T23:15:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 502 k 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 5.1 M 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-21T23:15:41.022 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-21T23:15:41.023 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 45 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.0.0-1643.g3bfc837d.el9 ceph 131 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-21T23:15:41.024 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-21T23:15:41.025 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-21T23:15:41.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-21T23:15:41.027 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-21T23:15:41.028 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-21T23:15:41.029 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-02-21T23:15:41.029 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-21T23:15:41.029 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:15:41.029 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-02-21T23:15:41.029 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================================= 2024-02-21T23:15:41.029 INFO:teuthology.orchestra.run.smithi133.stdout:Install 172 Packages 2024-02-21T23:15:41.029 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 16 Packages 2024-02-21T23:15:41.029 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:15:41.030 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 2.2 G 2024-02-21T23:15:41.030 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-02-21T23:15:41.846 INFO:teuthology.orchestra.run.smithi133.stdout:(1/188): ceph-19.0.0-1643.g3bfc837d.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-02-21T23:15:42.321 INFO:teuthology.orchestra.run.smithi133.stdout:(2/188): ceph-base-19.0.0-1643.g3bfc837d.el9.x8 7.0 MB/s | 5.3 MB 00:00 2024-02-21T23:15:46.637 INFO:teuthology.orchestra.run.smithi133.stdout:(3/188): ceph-base-debuginfo-19.0.0-1643.g3bfc8 13 MB/s | 68 MB 00:05 2024-02-21T23:15:48.312 INFO:teuthology.orchestra.run.smithi133.stdout:(4/188): ceph-debuginfo-19.0.0-1643.g3bfc837d.e 14 MB/s | 23 MB 00:01 2024-02-21T23:15:48.429 INFO:teuthology.orchestra.run.smithi133.stdout:(5/188): ceph-fuse-19.0.0-1643.g3bfc837d.el9.x8 7.5 MB/s | 889 kB 00:00 2024-02-21T23:15:48.971 INFO:teuthology.orchestra.run.smithi133.stdout:(6/188): ceph-fuse-debuginfo-19.0.0-1643.g3bfc8 11 MB/s | 6.0 MB 00:00 2024-02-21T23:15:49.038 INFO:teuthology.orchestra.run.smithi133.stdout:(7/188): ceph-immutable-object-cache-19.0.0-164 2.1 MB/s | 145 kB 00:00 2024-02-21T23:15:49.163 INFO:teuthology.orchestra.run.smithi133.stdout:(8/188): ceph-immutable-object-cache-debuginfo- 12 MB/s | 1.4 MB 00:00 2024-02-21T23:15:50.146 INFO:teuthology.orchestra.run.smithi133.stdout:(9/188): ceph-mds-19.0.0-1643.g3bfc837d.el9.x86 2.3 MB/s | 2.3 MB 00:00 2024-02-21T23:15:52.927 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 170/204 2024-02-21T23:15:53.738 INFO:teuthology.orchestra.run.smithi133.stdout:(10/188): ceph-mds-debuginfo-19.0.0-1643.g3bfc8 15 MB/s | 52 MB 00:03 2024-02-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi133.stdout:(11/188): ceph-mgr-19.0.0-1643.g3bfc837d.el9.x8 11 MB/s | 1.5 MB 00:00 2024-02-21T23:15:55.347 INFO:teuthology.orchestra.run.smithi133.stdout:(12/188): ceph-mgr-debuginfo-19.0.0-1643.g3bfc8 14 MB/s | 20 MB 00:01 2024-02-21T23:15:55.672 INFO:teuthology.orchestra.run.smithi133.stdout:(13/188): ceph-mon-19.0.0-1643.g3bfc837d.el9.x8 14 MB/s | 4.7 MB 00:00 2024-02-21T23:16:01.488 INFO:teuthology.orchestra.run.smithi133.stdout:(14/188): ceph-common-19.0.0-1643.g3bfc837d.el9 1.0 MB/s | 20 MB 00:19 2024-02-21T23:16:02.322 INFO:teuthology.orchestra.run.smithi133.stdout:(15/188): ceph-mon-debuginfo-19.0.0-1643.g3bfc8 15 MB/s | 99 MB 00:06 2024-02-21T23:16:05.513 INFO:teuthology.orchestra.run.smithi133.stdout:(16/188): ceph-osd-19.0.0-1643.g3bfc837d.el9.x8 4.2 MB/s | 17 MB 00:04 2024-02-21T23:16:06.747 INFO:teuthology.orchestra.run.smithi133.stdout:(17/188): ceph-radosgw-19.0.0-1643.g3bfc837d.el 7.9 MB/s | 9.8 MB 00:01 2024-02-21T23:16:10.136 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 171/204 2024-02-21T23:16:32.610 INFO:teuthology.orchestra.run.smithi133.stdout:(18/188): ceph-common-debuginfo-19.0.0-1643.g3b 8.7 MB/s | 436 MB 00:50 2024-02-21T23:16:32.768 INFO:teuthology.orchestra.run.smithi133.stdout:(19/188): ceph-selinux-19.0.0-1643.g3bfc837d.el 159 kB/s | 25 kB 00:00 2024-02-21T23:16:44.583 INFO:teuthology.orchestra.run.smithi133.stdout:(20/188): ceph-radosgw-debuginfo-19.0.0-1643.g3 4.7 MB/s | 176 MB 00:37 2024-02-21T23:16:46.234 INFO:teuthology.orchestra.run.smithi133.stdout:(21/188): ceph-test-19.0.0-1643.g3bfc837d.el9.x 3.5 MB/s | 47 MB 00:13 2024-02-21T23:16:46.434 INFO:teuthology.orchestra.run.smithi133.stdout:(22/188): cephfs-mirror-19.0.0-1643.g3bfc837d.e 1.1 MB/s | 220 kB 00:00 2024-02-21T23:16:46.517 INFO:teuthology.orchestra.run.smithi133.stdout:(23/188): cephfs-shell-19.0.0-1643.g3bfc837d.el 326 kB/s | 27 kB 00:00 2024-02-21T23:16:46.584 INFO:teuthology.orchestra.run.smithi133.stdout:(24/188): libcephfs-devel-19.0.0-1643.g3bfc837d 470 kB/s | 31 kB 00:00 2024-02-21T23:16:46.768 INFO:teuthology.orchestra.run.smithi133.stdout:(25/188): libcephfs2-19.0.0-1643.g3bfc837d.el9. 4.0 MB/s | 745 kB 00:00 2024-02-21T23:16:48.176 INFO:teuthology.orchestra.run.smithi133.stdout:(26/188): libcephfs2-debuginfo-19.0.0-1643.g3bf 3.8 MB/s | 5.4 MB 00:01 2024-02-21T23:16:48.251 INFO:teuthology.orchestra.run.smithi133.stdout:(27/188): libcephsqlite-19.0.0-1643.g3bfc837d.e 2.1 MB/s | 163 kB 00:00 2024-02-21T23:16:48.310 INFO:teuthology.orchestra.run.smithi133.stdout:(28/188): librados-devel-19.0.0-1643.g3bfc837d. 2.1 MB/s | 127 kB 00:00 2024-02-21T23:16:48.377 INFO:teuthology.orchestra.run.smithi133.stdout:(29/188): librados-devel-debuginfo-19.0.0-1643. 1.6 MB/s | 109 kB 00:00 2024-02-21T23:16:55.301 INFO:teuthology.orchestra.run.smithi133.stdout:(30/188): librados2-debuginfo-19.0.0-1643.g3bfc 6.5 MB/s | 45 MB 00:06 2024-02-21T23:16:55.410 INFO:teuthology.orchestra.run.smithi133.stdout:(31/188): libradosstriper1-19.0.0-1643.g3bfc837 4.5 MB/s | 502 kB 00:00 2024-02-21T23:17:01.734 INFO:teuthology.orchestra.run.smithi133.stdout:(32/188): ceph-osd-debuginfo-19.0.0-1643.g3bfc8 4.9 MB/s | 288 MB 00:59 2024-02-21T23:17:02.317 INFO:teuthology.orchestra.run.smithi133.stdout:(33/188): librgw2-19.0.0-1643.g3bfc837d.el9.x86 8.8 MB/s | 5.1 MB 00:00 2024-02-21T23:17:02.376 INFO:teuthology.orchestra.run.smithi133.stdout:(34/188): python3-ceph-argparse-19.0.0-1643.g3b 771 kB/s | 45 kB 00:00 2024-02-21T23:17:02.443 INFO:teuthology.orchestra.run.smithi133.stdout:(35/188): python3-ceph-common-19.0.0-1643.g3bfc 1.9 MB/s | 131 kB 00:00 2024-02-21T23:17:02.510 INFO:teuthology.orchestra.run.smithi133.stdout:(36/188): python3-cephfs-19.0.0-1643.g3bfc837d. 2.4 MB/s | 161 kB 00:00 2024-02-21T23:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout:(37/188): python3-cephfs-debuginfo-19.0.0-1643. 7.2 MB/s | 615 kB 00:00 2024-02-21T23:17:02.669 INFO:teuthology.orchestra.run.smithi133.stdout:(38/188): python3-rados-19.0.0-1643.g3bfc837d.e 4.2 MB/s | 321 kB 00:00 2024-02-21T23:17:03.128 INFO:teuthology.orchestra.run.smithi133.stdout:(39/188): librbd1-debuginfo-19.0.0-1643.g3bfc83 6.7 MB/s | 52 MB 00:07 2024-02-21T23:17:03.162 INFO:teuthology.orchestra.run.smithi133.stdout:(40/188): python3-rados-debuginfo-19.0.0-1643.g 1.5 MB/s | 770 kB 00:00 2024-02-21T23:17:03.195 INFO:teuthology.orchestra.run.smithi133.stdout:(41/188): python3-rbd-19.0.0-1643.g3bfc837d.el9 4.4 MB/s | 298 kB 00:00 2024-02-21T23:17:03.262 INFO:teuthology.orchestra.run.smithi133.stdout:(42/188): python3-rgw-19.0.0-1643.g3bfc837d.el9 1.5 MB/s | 100 kB 00:00 2024-02-21T23:17:03.295 INFO:teuthology.orchestra.run.smithi133.stdout:(43/188): python3-rbd-debuginfo-19.0.0-1643.g3b 8.4 MB/s | 1.1 MB 00:00 2024-02-21T23:17:03.321 INFO:teuthology.orchestra.run.smithi133.stdout:(44/188): python3-rgw-debuginfo-19.0.0-1643.g3b 5.6 MB/s | 337 kB 00:00 2024-02-21T23:17:03.346 INFO:teuthology.orchestra.run.smithi133.stdout:(45/188): rbd-fuse-19.0.0-1643.g3bfc837d.el9.x8 1.7 MB/s | 85 kB 00:00 2024-02-21T23:17:03.396 INFO:teuthology.orchestra.run.smithi133.stdout:(46/188): rbd-fuse-debuginfo-19.0.0-1643.g3bfc8 6.7 MB/s | 516 kB 00:00 2024-02-21T23:17:03.630 INFO:teuthology.orchestra.run.smithi133.stdout:(47/188): rbd-mirror-19.0.0-1643.g3bfc837d.el9. 11 MB/s | 3.1 MB 00:00 2024-02-21T23:17:03.705 INFO:teuthology.orchestra.run.smithi133.stdout:(48/188): rbd-nbd-19.0.0-1643.g3bfc837d.el9.x86 2.2 MB/s | 170 kB 00:00 2024-02-21T23:17:03.881 INFO:teuthology.orchestra.run.smithi133.stdout:(49/188): rbd-nbd-debuginfo-19.0.0-1643.g3bfc83 7.3 MB/s | 1.3 MB 00:00 2024-02-21T23:17:03.947 INFO:teuthology.orchestra.run.smithi133.stdout:(50/188): ceph-grafana-dashboards-19.0.0-1643.g 473 kB/s | 31 kB 00:00 2024-02-21T23:17:04.023 INFO:teuthology.orchestra.run.smithi133.stdout:(51/188): ceph-mgr-cephadm-19.0.0-1643.g3bfc837 1.8 MB/s | 138 kB 00:00 2024-02-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi133.stdout:(52/188): ceph-mgr-dashboard-19.0.0-1643.g3bfc8 8.6 MB/s | 1.8 MB 00:00 2024-02-21T23:17:05.048 INFO:teuthology.orchestra.run.smithi133.stdout:(53/188): ceph-mgr-diskprediction-local-19.0.0- 9.1 MB/s | 7.4 MB 00:00 2024-02-21T23:17:05.123 INFO:teuthology.orchestra.run.smithi133.stdout:(54/188): ceph-mgr-modules-core-19.0.0-1643.g3b 3.2 MB/s | 248 kB 00:00 2024-02-21T23:17:05.182 INFO:teuthology.orchestra.run.smithi133.stdout:(55/188): ceph-mgr-rook-19.0.0-1643.g3bfc837d.e 850 kB/s | 49 kB 00:00 2024-02-21T23:17:05.241 INFO:teuthology.orchestra.run.smithi133.stdout:(56/188): ceph-prometheus-alerts-19.0.0-1643.g3 262 kB/s | 15 kB 00:00 2024-02-21T23:17:05.316 INFO:teuthology.orchestra.run.smithi133.stdout:(57/188): ceph-volume-19.0.0-1643.g3bfc837d.el9 3.4 MB/s | 264 kB 00:00 2024-02-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi133.stdout:(58/188): cephadm-19.0.0-1643.g3bfc837d.el9.noa 5.9 MB/s | 748 kB 00:00 2024-02-21T23:17:05.508 INFO:teuthology.orchestra.run.smithi133.stdout:(59/188): cephfs-top-19.0.0-1643.g3bfc837d.el9. 273 kB/s | 18 kB 00:00 2024-02-21T23:17:05.619 INFO:teuthology.orchestra.run.smithi133.stdout:(60/188): libaio-devel-0.3.112-1.el8.x86_64.rpm 159 kB/s | 18 kB 00:00 2024-02-21T23:17:06.342 INFO:teuthology.orchestra.run.smithi133.stdout:(61/188): fuse-2.9.9-15.el9.x86_64.rpm 111 kB/s | 80 kB 00:00 2024-02-21T23:17:07.267 INFO:teuthology.orchestra.run.smithi133.stdout:(62/188): jq-1.6-16.el9.x86_64.rpm 202 kB/s | 187 kB 00:00 2024-02-21T23:17:08.051 INFO:teuthology.orchestra.run.smithi133.stdout:(63/188): rbd-mirror-debuginfo-19.0.0-1643.g3bf 11 MB/s | 52 MB 00:04 2024-02-21T23:17:08.934 INFO:teuthology.orchestra.run.smithi133.stdout:(64/188): libquadmath-11.4.1-3.el9.x86_64.rpm 221 kB/s | 195 kB 00:00 2024-02-21T23:17:09.251 INFO:teuthology.orchestra.run.smithi133.stdout:(65/188): libgfortran-11.4.1-3.el9.x86_64.rpm 405 kB/s | 803 kB 00:01 2024-02-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi133.stdout:(66/188): oniguruma-6.9.6-1.el9.5.x86_64.rpm 245 kB/s | 218 kB 00:00 2024-02-21T23:17:10.210 INFO:teuthology.orchestra.run.smithi133.stdout:(67/188): python3-cffi-1.14.5-5.el9.x86_64.rpm 264 kB/s | 253 kB 00:00 2024-02-21T23:17:11.010 INFO:teuthology.orchestra.run.smithi133.stdout:(68/188): python3-ply-3.11-14.el9.noarch.rpm 133 kB/s | 106 kB 00:00 2024-02-21T23:17:11.993 INFO:teuthology.orchestra.run.smithi133.stdout:(69/188): python3-pycparser-2.20-6.el9.noarch.r 137 kB/s | 135 kB 00:00 2024-02-21T23:17:12.244 INFO:teuthology.orchestra.run.smithi133.stdout:(70/188): python3-cryptography-36.0.1-4.el9.x86 519 kB/s | 1.2 MB 00:02 2024-02-21T23:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:(71/188): python3-requests-2.25.1-8.el9.noarch. 154 kB/s | 125 kB 00:00 2024-02-21T23:17:13.152 INFO:teuthology.orchestra.run.smithi133.stdout:(72/188): python3-urllib3-1.26.5-5.el9.noarch.r 243 kB/s | 215 kB 00:00 2024-02-21T23:17:14.344 INFO:teuthology.orchestra.run.smithi133.stdout:(73/188): automake-1.16.2-8.el9.noarch.rpm 447 kB/s | 689 kB 00:01 2024-02-21T23:17:14.944 INFO:teuthology.orchestra.run.smithi133.stdout:(74/188): bison-3.7.4-5.el9.x86_64.rpm 527 kB/s | 943 kB 00:01 2024-02-21T23:17:15.111 INFO:teuthology.orchestra.run.smithi133.stdout:(75/188): boost-program-options-1.75.0-8.el9.x8 139 kB/s | 107 kB 00:00 2024-02-21T23:17:15.570 INFO:teuthology.orchestra.run.smithi133.stdout:(76/188): elfutils-libelf-devel-0.190-2.el9.x86 34 kB/s | 21 kB 00:00 2024-02-21T23:17:16.220 INFO:teuthology.orchestra.run.smithi133.stdout:(77/188): flexiblas-3.0.4-8.el9.x86_64.rpm 50 kB/s | 33 kB 00:00 2024-02-21T23:17:16.245 INFO:teuthology.orchestra.run.smithi133.stdout:(78/188): flex-2.6.4-9.el9.x86_64.rpm 277 kB/s | 313 kB 00:01 2024-02-21T23:17:16.796 INFO:teuthology.orchestra.run.smithi133.stdout:(79/188): flexiblas-openblas-openmp-3.0.4-8.el9 33 kB/s | 18 kB 00:00 2024-02-21T23:17:17.506 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-debuginfo-2:19.0.0-1643.g3bfc837d.e 172/204 2024-02-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi133.stdout:(80/188): libpmemobj-1.12.1-1.el9.x86_64.rpm 154 kB/s | 160 kB 00:01 2024-02-21T23:17:18.301 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 173/204 2024-02-21T23:17:18.538 INFO:teuthology.orchestra.run.smithi133.stdout:(81/188): librabbitmq-0.11.0-7.el9.x86_64.rpm 65 kB/s | 45 kB 00:00 2024-02-21T23:17:18.549 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-debuginfo-2:19.0.0-1643.g3bfc837d.el9 174/204 2024-02-21T23:17:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:(82/188): librdkafka-1.6.1-102.el9.x86_64.rpm 457 kB/s | 662 kB 00:01 2024-02-21T23:17:21.004 INFO:teuthology.orchestra.run.smithi133.stdout:(83/188): flexiblas-netlib-3.0.4-8.el9.x86_64.r 640 kB/s | 3.0 MB 00:04 2024-02-21T23:17:21.388 INFO:teuthology.orchestra.run.smithi133.stdout:(84/188): libtool-2.4.6-45.el9.x86_64.rpm 416 kB/s | 582 kB 00:01 2024-02-21T23:17:21.621 INFO:teuthology.orchestra.run.smithi133.stdout:(85/188): libuuid-devel-2.37.4-15.el9.x86_64.rp 41 kB/s | 26 kB 00:00 2024-02-21T23:17:22.047 INFO:teuthology.orchestra.run.smithi133.stdout:(86/188): libzstd-devel-1.5.1-2.el9.x86_64.rpm 72 kB/s | 47 kB 00:00 2024-02-21T23:17:22.680 INFO:teuthology.orchestra.run.smithi133.stdout:(87/188): lttng-ust-2.12.0-6.el9.x86_64.rpm 283 kB/s | 292 kB 00:01 2024-02-21T23:17:22.905 INFO:teuthology.orchestra.run.smithi133.stdout:(88/188): lua-5.4.4-4.el9.x86_64.rpm 220 kB/s | 188 kB 00:00 2024-02-21T23:17:23.306 INFO:teuthology.orchestra.run.smithi133.stdout:(89/188): openblas-0.3.21-2.el9.x86_64.rpm 56 kB/s | 35 kB 00:00 2024-02-21T23:17:23.540 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-debuginfo-2:19.0.0-1643.g3bfc837d 175/204 2024-02-21T23:17:29.167 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados2-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 176/204 2024-02-21T23:17:29.261 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librbd1-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x8 177/204 2024-02-21T23:17:29.360 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-debuginfo-2:19.0.0-1643.g3bfc837d 178/204 2024-02-21T23:17:29.363 INFO:teuthology.orchestra.run.smithi133.stdout:(90/188): openssl-devel-3.0.7-25.el9.x86_64.rpm 697 kB/s | 4.1 MB 00:06 2024-02-21T23:17:29.496 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-debuginfo-2:19.0.0-1643.g3bfc837d. 179/204 2024-02-21T23:17:29.545 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-debuginfo-2:19.0.0-1643.g3bfc837d.el 180/204 2024-02-21T23:17:29.635 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-debuginfo-2:19.0.0-1643.g3bfc837d.el 181/204 2024-02-21T23:17:29.947 INFO:teuthology.orchestra.run.smithi133.stdout:(91/188): openblas-openmp-0.3.21-2.el9.x86_64.r 689 kB/s | 4.7 MB 00:07 2024-02-21T23:17:31.880 INFO:teuthology.orchestra.run.smithi133.stdout:(92/188): postgresql-contrib-13.11-1.el9.x86_64 469 kB/s | 905 kB 00:01 2024-02-21T23:17:32.339 INFO:teuthology.orchestra.run.smithi133.stdout:(93/188): postgresql-13.11-1.el9.x86_64.rpm 545 kB/s | 1.6 MB 00:02 2024-02-21T23:17:32.856 INFO:teuthology.orchestra.run.smithi133.stdout:(94/188): postgresql-private-libs-13.11-1.el9.x 138 kB/s | 135 kB 00:00 2024-02-21T23:17:34.881 INFO:teuthology.orchestra.run.smithi133.stdout:(95/188): protobuf-3.14.0-13.el9.x86_64.rpm 510 kB/s | 1.0 MB 00:02 2024-02-21T23:17:35.193 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 182/204 2024-02-21T23:17:35.358 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-debuginfo-2:19.0.0-1643.g3bfc837d.el9 183/204 2024-02-21T23:17:35.606 INFO:teuthology.orchestra.run.smithi133.stdout:(96/188): python3-attrs-20.3.0-7.el9.noarch.rpm 121 kB/s | 87 kB 00:00 2024-02-21T23:17:36.174 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x8 184/204 2024-02-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 185/204 2024-02-21T23:17:36.388 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 186/204 2024-02-21T23:17:36.430 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 187/204 2024-02-21T23:17:36.455 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 188/204 2024-02-21T23:17:36.480 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 189/204 2024-02-21T23:17:36.506 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 190/204 2024-02-21T23:17:36.532 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 191/204 2024-02-21T23:17:36.532 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 192/204 2024-02-21T23:17:36.619 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 192/204 2024-02-21T23:17:36.666 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 193/204 2024-02-21T23:17:36.692 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 194/204 2024-02-21T23:17:36.717 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 195/204 2024-02-21T23:17:36.743 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 196/204 2024-02-21T23:17:36.743 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 197/204 2024-02-21T23:17:36.827 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 197/204 2024-02-21T23:17:36.852 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 198/204 2024-02-21T23:17:36.857 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 199/204 2024-02-21T23:17:36.858 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 199/204 2024-02-21T23:17:37.038 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 199/204 2024-02-21T23:17:37.038 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 200/204 2024-02-21T23:17:37.122 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 200/204 2024-02-21T23:17:37.146 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 201/204 2024-02-21T23:17:37.175 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 202/204 2024-02-21T23:17:37.206 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 203/204 2024-02-21T23:17:37.272 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 204/204 2024-02-21T23:17:40.514 INFO:teuthology.orchestra.run.smithi133.stdout:(97/188): postgresql-server-13.11-1.el9.x86_64. 720 kB/s | 5.7 MB 00:08 2024-02-21T23:17:41.514 INFO:teuthology.orchestra.run.smithi133.stdout:(98/188): python3-devel-3.9.17-2.el9.x86_64.rpm 245 kB/s | 245 kB 00:00 2024-02-21T23:17:42.539 INFO:teuthology.orchestra.run.smithi133.stdout:(99/188): python3-jinja2-2.11.3-5.el9.noarch.rp 242 kB/s | 248 kB 00:01 2024-02-21T23:17:43.156 INFO:teuthology.orchestra.run.smithi133.stdout:(100/188): python3-jsonpatch-1.21-16.el9.noarch 42 kB/s | 26 kB 00:00 2024-02-21T23:17:43.714 INFO:teuthology.orchestra.run.smithi133.stdout:(101/188): python3-jsonpointer-2.0-4.el9.noarch 34 kB/s | 19 kB 00:00 2024-02-21T23:17:43.881 INFO:teuthology.orchestra.run.smithi133.stdout:(102/188): python3-babel-2.9.1-2.el9.noarch.rpm 738 kB/s | 6.0 MB 00:08 2024-02-21T23:17:44.531 INFO:teuthology.orchestra.run.smithi133.stdout:(103/188): python3-markupsafe-1.1.1-12.el9.x86_ 53 kB/s | 35 kB 00:00 2024-02-21T23:17:44.632 INFO:teuthology.orchestra.run.smithi133.stdout:(104/188): python3-mako-1.1.4-6.el9.noarch.rpm 188 kB/s | 172 kB 00:00 2024-02-21T23:17:46.007 INFO:teuthology.orchestra.run.smithi133.stdout:(105/188): python3-numpy-f2py-1.20.1-5.el9.x86_ 182 kB/s | 250 kB 00:01 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 204/204 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:19.0.0-1643.g3bfc837d.el9.x86_64 1/204 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:19.0.0-1643.g3bfc837d.el9.x86_64 2/204 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 3/204 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 4/204 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-debuginfo-2:19.0.0-1643.g3bfc837d.el 5/204 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_6 6/204 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 7/204 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 8/204 2024-02-21T23:17:46.324 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1643.g3bfc8 9/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.0.0-1 10/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:19.0.0-1643.g3bfc837d.el9.x86_64 11/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 12/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:19.0.0-1643.g3bfc837d.el9.x86_64 13/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 14/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:19.0.0-1643.g3bfc837d.el9.x86_64 15/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 16/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:19.0.0-1643.g3bfc837d.el9.x86_64 17/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 18/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 19/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-debuginfo-2:19.0.0-1643.g3bfc837d.e 20/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:19.0.0-1643.g3bfc837d.el9.x86_64 21/204 2024-02-21T23:17:46.325 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:19.0.0-1643.g3bfc837d.el9.x86_64 22/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 23/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 24/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-shell-2:19.0.0-1643.g3bfc837d.el9.x86_64 25/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:19.0.0-1643.g3bfc837d.el9.x86_ 26/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:19.0.0-1643.g3bfc837d.el9.x86_64 27/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-debuginfo-2:19.0.0-1643.g3bfc837d.el9 28/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:19.0.0-1643.g3bfc837d.el9.x86_64 29/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:19.0.0-1643.g3bfc837d.el9.x86_6 30/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-debuginfo-2:19.0.0-1643.g3bfc837d 31/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 32/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:19.0.0-1643.g3bfc837d.el9.x86 33/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x8 34/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:19.0.0-1643.g3bfc837d.el9.x86_64 35/204 2024-02-21T23:17:46.326 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1643.g3bfc837d.el 36/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:19.0.0-1643.g3bfc837d.el9. 37/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:19.0.0-1643.g3bfc837d.el9.x86_6 38/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-debuginfo-2:19.0.0-1643.g3bfc837d 39/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:19.0.0-1643.g3bfc837d.el9.x86_64 40/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-debuginfo-2:19.0.0-1643.g3bfc837d. 41/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 42/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-debuginfo-2:19.0.0-1643.g3bfc837d.el 43/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 44/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-debuginfo-2:19.0.0-1643.g3bfc837d.el 45/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 46/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 47/204 2024-02-21T23:17:46.327 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 48/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-debuginfo-2:19.0.0-1643.g3bfc837d.el9 49/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 50/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x8 51/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1643.g3bfc837d. 52/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1643.g3bfc837d.el9.noa 53/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1643.g3bfc837d.el9.n 54/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1643.g3bf 55/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1643.g3bfc837d.el 56/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1643.g3bfc837d.el9.noarch 57/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1643.g3bfc837d.e 58/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:19.0.0-1643.g3bfc837d.el9.noarch 59/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 60/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:19.0.0-1643.g3bfc837d.el9.noarch 61/204 2024-02-21T23:17:46.328 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 62/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 63/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : jq-1.6-16.el9.x86_64 64/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 65/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 66/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 67/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 68/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 69/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 70/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 71/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 72/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 73/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 74/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 75/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 76/204 2024-02-21T23:17:46.329 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 77/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 78/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 79/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 80/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 82/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 83/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 84/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 85/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 86/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 87/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 88/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-5.4.4-4.el9.x86_64 89/204 2024-02-21T23:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 90/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 91/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 92/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-13.11-1.el9.x86_64 93/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 94/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 95/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 96/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 97/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 98/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 99/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 100/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 101/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 102/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 103/204 2024-02-21T23:17:46.331 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 104/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 105/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 106/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 107/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 108/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 109/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 110/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 111/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 112/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 113/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 114/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 115/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 116/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 117/204 2024-02-21T23:17:46.332 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 118/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 119/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 120/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 121/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 122/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 123/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 124/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 125/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 126/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 127/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 128/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 129/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 130/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 131/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 132/204 2024-02-21T23:17:46.333 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 133/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 134/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 135/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 136/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 137/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 138/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 139/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 140/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 141/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 142/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 143/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 144/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 145/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 146/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 147/204 2024-02-21T23:17:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 148/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 149/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 150/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 151/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 152/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 153/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 154/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 155/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 156/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 157/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 158/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 159/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 160/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 161/204 2024-02-21T23:17:46.335 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 162/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 163/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 164/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 165/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 166/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 167/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 168/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 169/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 170/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-3.el9.x86_64 171/204 2024-02-21T23:17:46.336 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 172/204 2024-02-21T23:17:46.339 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:19.0.0-1643.g3bfc837d.el9.x86_64 173/204 2024-02-21T23:17:46.339 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 174/204 2024-02-21T23:17:46.339 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:19.0.0-1643.g3bfc837d.el9.x86_64 175/204 2024-02-21T23:17:46.339 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 176/204 2024-02-21T23:17:46.339 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 177/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 178/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 179/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 180/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 181/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 182/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 183/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 184/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : acl-2.3.1-4.el9.x86_64 185/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : acl-2.3.1-3.el9.x86_64 186/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 187/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 188/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 189/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 190/204 2024-02-21T23:17:46.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 191/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 192/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 193/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 194/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 195/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 196/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 197/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 198/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/204 2024-02-21T23:17:46.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 203/204 2024-02-21T23:17:46.923 INFO:teuthology.orchestra.run.smithi133.stdout:(106/188): python3-oauthlib-3.1.1-5.el9.noarch. 242 kB/s | 222 kB 00:00 2024-02-21T23:17:47.632 INFO:teuthology.orchestra.run.smithi133.stdout:(107/188): python3-packaging-20.9-5.el9.noarch. 109 kB/s | 77 kB 00:00 2024-02-21T23:17:48.316 INFO:teuthology.orchestra.run.smithi133.stdout:(108/188): python3-prettytable-0.7.2-27.el9.noa 62 kB/s | 42 kB 00:00 2024-02-21T23:17:49.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 204/204 2024-02-21T23:17:49.344 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:17:49.344 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-02-21T23:17:49.344 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-21T23:17:49.344 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-21T23:17:49.344 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-21T23:17:49.344 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-21T23:17:49.344 INFO:teuthology.orchestra.run.smithi142.stdout: acl-2.3.1-4.el9.x86_64 2024-02-21T23:17:49.344 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-21T23:17:49.345 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.346 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-21T23:17:49.349 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-21T23:17:49.350 INFO:teuthology.orchestra.run.smithi142.stdout: jq-1.6-16.el9.x86_64 2024-02-21T23:17:49.350 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-21T23:17:49.350 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-21T23:17:49.350 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-21T23:17:49.350 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.350 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.350 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.350 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.350 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-21T23:17:49.351 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: lua-5.4.4-4.el9.x86_64 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-21T23:17:49.352 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-13.11-1.el9.x86_64 2024-02-21T23:17:49.353 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-21T23:17:49.353 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-21T23:17:49.353 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-21T23:17:49.353 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-21T23:17:49.353 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-21T23:17:49.353 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-21T23:17:49.354 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-21T23:17:49.355 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-21T23:17:49.355 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-21T23:17:49.355 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-21T23:17:49.355 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-21T23:17:49.355 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-21T23:17:49.355 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-21T23:17:49.355 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-21T23:17:49.355 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-21T23:17:49.355 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-21T23:17:49.356 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-21T23:17:49.357 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-21T23:17:49.357 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-21T23:17:49.357 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-21T23:17:49.357 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-21T23:17:49.357 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-02-21T23:17:49.357 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-21T23:17:49.357 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-21T23:17:49.357 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-21T23:17:49.357 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.358 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-21T23:17:49.359 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-3.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-21T23:17:49.360 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-21T23:17:49.361 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-21T23:17:49.361 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:17:49.361 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-02-21T23:17:49.574 INFO:teuthology.orchestra.run.smithi133.stdout:(109/188): python3-protobuf-3.14.0-13.el9.noarc 214 kB/s | 269 kB 00:01 2024-02-21T23:17:50.053 DEBUG:teuthology.parallel:result is None 2024-02-21T23:17:50.399 INFO:teuthology.orchestra.run.smithi133.stdout:(110/188): python3-pyasn1-0.4.8-6.el9.noarch.rp 193 kB/s | 159 kB 00:00 2024-02-21T23:17:51.416 INFO:teuthology.orchestra.run.smithi133.stdout:(111/188): python3-pyasn1-modules-0.4.8-6.el9.n 274 kB/s | 279 kB 00:01 2024-02-21T23:17:52.100 INFO:teuthology.orchestra.run.smithi133.stdout:(112/188): python3-pytz-2021.1-5.el9.noarch.rpm 75 kB/s | 51 kB 00:00 2024-02-21T23:17:52.258 INFO:teuthology.orchestra.run.smithi133.stdout:(113/188): python3-numpy-1.20.1-5.el9.x86_64.rp 676 kB/s | 5.1 MB 00:07 2024-02-21T23:17:52.775 INFO:teuthology.orchestra.run.smithi133.stdout:(114/188): python3-requests-oauthlib-1.3.0-12.e 79 kB/s | 54 kB 00:00 2024-02-21T23:17:53.409 INFO:teuthology.orchestra.run.smithi133.stdout:(115/188): python3-toml-0.10.2-6.el9.noarch.rpm 66 kB/s | 42 kB 00:00 2024-02-21T23:17:54.042 INFO:teuthology.orchestra.run.smithi133.stdout:(116/188): python3-wcwidth-0.2.5-8.el9.noarch.r 70 kB/s | 44 kB 00:00 2024-02-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi133.stdout:(117/188): qatlib-23.11.0-1.el9.x86_64.rpm 215 kB/s | 216 kB 00:01 2024-02-21T23:17:55.726 INFO:teuthology.orchestra.run.smithi133.stdout:(118/188): qatlib-service-23.11.0-1.el9.x86_64. 52 kB/s | 35 kB 00:00 2024-02-21T23:17:56.393 INFO:teuthology.orchestra.run.smithi133.stdout:(119/188): qatzip-libs-1.1.2-2.el9.x86_64.rpm 67 kB/s | 44 kB 00:00 2024-02-21T23:17:57.401 INFO:teuthology.orchestra.run.smithi133.stdout:(120/188): socat-1.7.4.1-5.el9.x86_64.rpm 302 kB/s | 305 kB 00:01 2024-02-21T23:17:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:(121/188): uuid-1.6.2-55.el9.x86_64.rpm 75 kB/s | 58 kB 00:00 2024-02-21T23:17:59.051 INFO:teuthology.orchestra.run.smithi133.stdout:(122/188): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 106 kB/s | 94 kB 00:00 2024-02-21T23:17:59.768 INFO:teuthology.orchestra.run.smithi133.stdout:(123/188): xmlstarlet-1.6.1-20.el9.x86_64.rpm 89 kB/s | 64 kB 00:00 2024-02-21T23:18:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:(124/188): zlib-devel-1.2.11-40.el9.x86_64.rpm 69 kB/s | 45 kB 00:00 2024-02-21T23:18:00.502 INFO:teuthology.orchestra.run.smithi133.stdout:(125/188): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-02-21T23:18:00.652 INFO:teuthology.orchestra.run.smithi133.stdout:(126/188): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-02-21T23:18:02.019 INFO:teuthology.orchestra.run.smithi133.stdout:(127/188): abseil-cpp-20211102.0-3.el9.x86_64.r 401 kB/s | 548 kB 00:01 2024-02-21T23:18:02.894 INFO:teuthology.orchestra.run.smithi133.stdout:(128/188): gperftools-libs-2.9.1-2.el9.x86_64.r 353 kB/s | 309 kB 00:00 2024-02-21T23:18:03.353 INFO:teuthology.orchestra.run.smithi133.stdout:(129/188): grpc-data-1.46.7-10.el9.noarch.rpm 42 kB/s | 19 kB 00:00 2024-02-21T23:18:07.927 INFO:teuthology.orchestra.run.smithi133.stdout:(130/188): libarrow-9.0.0-10.el9.x86_64.rpm 989 kB/s | 4.4 MB 00:04 2024-02-21T23:18:08.319 INFO:teuthology.orchestra.run.smithi133.stdout:(131/188): libarrow-doc-9.0.0-10.el9.noarch.rpm 66 kB/s | 26 kB 00:00 2024-02-21T23:18:08.786 INFO:teuthology.orchestra.run.smithi133.stdout:(132/188): liboath-2.6.7-2.el9.x86_64.rpm 104 kB/s | 49 kB 00:00 2024-02-21T23:18:09.336 INFO:teuthology.orchestra.run.smithi133.stdout:(133/188): libunwind-1.6.2-1.el9.x86_64.rpm 123 kB/s | 67 kB 00:00 2024-02-21T23:18:09.995 INFO:teuthology.orchestra.run.smithi133.stdout:(134/188): luarocks-3.9.2-1.el9.noarch.rpm 230 kB/s | 151 kB 00:00 2024-02-21T23:18:11.778 INFO:teuthology.orchestra.run.smithi133.stdout:(135/188): parquet-libs-9.0.0-10.el9.x86_64.rpm 470 kB/s | 838 kB 00:01 2024-02-21T23:18:12.728 INFO:teuthology.orchestra.run.smithi133.stdout:(136/188): python3-asyncssh-2.13.2-4.el9.noarch 577 kB/s | 548 kB 00:00 2024-02-21T23:18:13.187 INFO:teuthology.orchestra.run.smithi133.stdout:(137/188): python3-bcrypt-3.2.2-1.el9.x86_64.rp 95 kB/s | 43 kB 00:00 2024-02-21T23:18:13.654 INFO:teuthology.orchestra.run.smithi133.stdout:(138/188): python3-cachetools-4.2.4-1.el9.noarc 69 kB/s | 32 kB 00:00 2024-02-21T23:18:14.046 INFO:teuthology.orchestra.run.smithi133.stdout:(139/188): python3-certifi-2021.10.8-2.el9.noar 38 kB/s | 15 kB 00:00 2024-02-21T23:18:14.671 INFO:teuthology.orchestra.run.smithi133.stdout:(140/188): python3-cheroot-8.6.0-4.el9.noarch.r 275 kB/s | 172 kB 00:00 2024-02-21T23:18:15.538 INFO:teuthology.orchestra.run.smithi133.stdout:(141/188): python3-scipy-1.6.2-8.el9.x86_64.rpm 715 kB/s | 16 MB 00:23 2024-02-21T23:18:15.563 INFO:teuthology.orchestra.run.smithi133.stdout:(142/188): python3-cherrypy-18.6.1-2.el9.noarch 402 kB/s | 358 kB 00:00 2024-02-21T23:18:16.018 INFO:teuthology.orchestra.run.smithi133.stdout:(143/188): python3-colorama-0.4.4-12.el9.noarch 78 kB/s | 35 kB 00:00 2024-02-21T23:18:16.313 INFO:teuthology.orchestra.run.smithi133.stdout:(144/188): python3-cmd2-2.4.2-2.el9.noarch.rpm 391 kB/s | 303 kB 00:00 2024-02-21T23:18:16.697 INFO:teuthology.orchestra.run.smithi133.stdout:(145/188): python3-google-auth-2.27.0-1.el9.noa 297 kB/s | 203 kB 00:00 2024-02-21T23:18:17.306 INFO:teuthology.orchestra.run.smithi133.stdout:(146/188): python3-grpcio-tools-1.46.7-10.el9.x 238 kB/s | 144 kB 00:00 2024-02-21T23:18:17.681 INFO:teuthology.orchestra.run.smithi133.stdout:(147/188): python3-jaraco-8.2.1-3.el9.noarch.rp 28 kB/s | 11 kB 00:00 2024-02-21T23:18:18.056 INFO:teuthology.orchestra.run.smithi133.stdout:(148/188): python3-jaraco-classes-3.2.1-5.el9.n 47 kB/s | 18 kB 00:00 2024-02-21T23:18:18.248 INFO:teuthology.orchestra.run.smithi133.stdout:(149/188): python3-grpcio-1.46.7-10.el9.x86_64. 1.1 MB/s | 2.0 MB 00:01 2024-02-21T23:18:18.440 INFO:teuthology.orchestra.run.smithi133.stdout:(150/188): python3-jaraco-collections-3.0.0-8.e 60 kB/s | 23 kB 00:00 2024-02-21T23:18:18.632 INFO:teuthology.orchestra.run.smithi133.stdout:(151/188): python3-jaraco-functools-3.5.0-2.el9 51 kB/s | 19 kB 00:00 2024-02-21T23:18:18.849 INFO:teuthology.orchestra.run.smithi133.stdout:(152/188): python3-jaraco-text-3.2.0-6.el9.noar 48 kB/s | 20 kB 00:00 2024-02-21T23:18:19.299 INFO:teuthology.orchestra.run.smithi133.stdout:(153/188): python3-logutils-0.3.5-21.el9.noarch 103 kB/s | 46 kB 00:00 2024-02-21T23:18:19.836 INFO:teuthology.orchestra.run.smithi133.stdout:(154/188): python3-more-itertools-8.12.0-2.el9. 147 kB/s | 79 kB 00:00 2024-02-21T23:18:19.870 INFO:teuthology.orchestra.run.smithi133.stdout:(155/188): python3-kubernetes-26.1.0-2.el9.noar 846 kB/s | 1.0 MB 00:01 2024-02-21T23:18:20.320 INFO:teuthology.orchestra.run.smithi133.stdout:(156/188): python3-natsort-7.1.1-5.el9.noarch.r 119 kB/s | 58 kB 00:00 2024-02-21T23:18:20.637 INFO:teuthology.orchestra.run.smithi133.stdout:(157/188): python3-pecan-1.4.2-3.el9.noarch.rpm 355 kB/s | 272 kB 00:00 2024-02-21T23:18:20.712 INFO:teuthology.orchestra.run.smithi133.stdout:(158/188): python3-portend-3.1.0-2.el9.noarch.r 42 kB/s | 16 kB 00:00 2024-02-21T23:18:21.313 INFO:teuthology.orchestra.run.smithi133.stdout:(159/188): python3-pyperclip-1.8.2-2.el9.noarch 47 kB/s | 28 kB 00:00 2024-02-21T23:18:21.563 INFO:teuthology.orchestra.run.smithi133.stdout:(160/188): python3-pyOpenSSL-21.0.0-1.el9.noarc 97 kB/s | 90 kB 00:00 2024-02-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi133.stdout:(161/188): python3-repoze-lru-0.7-16.el9.noarch 49 kB/s | 31 kB 00:00 2024-02-21T23:18:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:(162/188): python3-routes-2.5.1-5.el9.noarch.rp 273 kB/s | 188 kB 00:00 2024-02-21T23:18:22.422 INFO:teuthology.orchestra.run.smithi133.stdout:(163/188): python3-rsa-4.9-2.el9.noarch.rpm 124 kB/s | 59 kB 00:00 2024-02-21T23:18:22.739 INFO:teuthology.orchestra.run.smithi133.stdout:(164/188): python3-tempora-5.0.0-2.el9.noarch.r 78 kB/s | 36 kB 00:00 2024-02-21T23:18:22.881 INFO:teuthology.orchestra.run.smithi133.stdout:(165/188): python3-typing-extensions-4.4.0-2.el 111 kB/s | 51 kB 00:00 2024-02-21T23:18:23.431 INFO:teuthology.orchestra.run.smithi133.stdout:(166/188): python3-websocket-client-1.2.3-2.el9 163 kB/s | 90 kB 00:00 2024-02-21T23:18:23.456 INFO:teuthology.orchestra.run.smithi133.stdout:(167/188): python3-webob-1.8.7-6.el9.noarch.rpm 320 kB/s | 230 kB 00:00 2024-02-21T23:18:23.831 INFO:teuthology.orchestra.run.smithi133.stdout:(168/188): python3-zc-lockfile-2.0-10.el9.noarc 53 kB/s | 20 kB 00:00 2024-02-21T23:18:24.282 INFO:teuthology.orchestra.run.smithi133.stdout:(169/188): python3-werkzeug-2.0.3-3.el9.1.noarc 502 kB/s | 427 kB 00:00 2024-02-21T23:18:24.532 INFO:teuthology.orchestra.run.smithi133.stdout:(170/188): re2-20211101-3.el9.x86_64.rpm 274 kB/s | 192 kB 00:00 2024-02-21T23:18:26.232 INFO:teuthology.orchestra.run.smithi133.stdout:(171/188): thrift-0.15.0-2.el9.x86_64.rpm 831 kB/s | 1.6 MB 00:01 2024-02-21T23:18:26.432 INFO:teuthology.orchestra.run.smithi133.stdout:(172/188): librados2-19.0.0-1643.g3bfc837d.el9. 1.8 MB/s | 3.4 MB 00:01 2024-02-21T23:18:28.357 INFO:teuthology.orchestra.run.smithi133.stdout:(173/188): librbd1-19.0.0-1643.g3bfc837d.el9.x8 1.5 MB/s | 3.1 MB 00:02 2024-02-21T23:18:30.391 INFO:teuthology.orchestra.run.smithi133.stdout:(174/188): NetworkManager-1.45.91-1.el9.x86_64. 588 kB/s | 2.3 MB 00:03 2024-02-21T23:18:31.032 INFO:teuthology.orchestra.run.smithi133.stdout:(175/188): NetworkManager-team-1.45.91-1.el9.x8 66 kB/s | 42 kB 00:00 2024-02-21T23:18:31.524 INFO:teuthology.orchestra.run.smithi133.stdout:(176/188): NetworkManager-libnm-1.45.91-1.el9.x 586 kB/s | 1.8 MB 00:03 2024-02-21T23:18:31.966 INFO:teuthology.orchestra.run.smithi133.stdout:(177/188): NetworkManager-tui-1.45.91-1.el9.x86 265 kB/s | 247 kB 00:00 2024-02-21T23:18:32.283 INFO:teuthology.orchestra.run.smithi133.stdout:(178/188): acl-2.3.1-4.el9.x86_64.rpm 94 kB/s | 71 kB 00:00 2024-02-21T23:18:32.608 INFO:teuthology.orchestra.run.smithi133.stdout:(179/188): elfutils-debuginfod-client-0.190-2.e 57 kB/s | 36 kB 00:00 2024-02-21T23:18:33.208 INFO:teuthology.orchestra.run.smithi133.stdout:(180/188): elfutils-libelf-0.190-2.el9.x86_64.r 208 kB/s | 192 kB 00:00 2024-02-21T23:18:33.592 INFO:teuthology.orchestra.run.smithi133.stdout:(181/188): elfutils-libs-0.190-2.el9.x86_64.rpm 260 kB/s | 256 kB 00:00 2024-02-21T23:18:33.834 INFO:teuthology.orchestra.run.smithi133.stdout:(182/188): libacl-2.3.1-4.el9.x86_64.rpm 37 kB/s | 23 kB 00:00 2024-02-21T23:18:35.776 INFO:teuthology.orchestra.run.smithi133.stdout:(183/188): openssl-3.0.7-25.el9.x86_64.rpm 569 kB/s | 1.2 MB 00:02 2024-02-21T23:18:36.884 INFO:teuthology.orchestra.run.smithi133.stdout:(184/188): xfsdump-3.1.12-4.el9.x86_64.rpm 292 kB/s | 323 kB 00:01 2024-02-21T23:18:37.293 INFO:teuthology.orchestra.run.smithi133.stdout:(185/188): openssl-libs-3.0.7-25.el9.x86_64.rpm 643 kB/s | 2.2 MB 00:03 2024-02-21T23:18:38.043 INFO:teuthology.orchestra.run.smithi133.stdout:(186/188): libacl-devel-2.3.1-4.el9.x86_64.rpm 99 kB/s | 74 kB 00:00 2024-02-21T23:18:38.918 INFO:teuthology.orchestra.run.smithi133.stdout:(187/188): xfsprogs-6.3.0-1.el9.x86_64.rpm 541 kB/s | 1.1 MB 00:02 2024-02-21T23:19:06.231 INFO:teuthology.orchestra.run.smithi133.stdout:(188/188): ceph-test-debuginfo-19.0.0-1643.g3bf 4.8 MB/s | 675 MB 02:21 2024-02-21T23:19:06.235 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-02-21T23:19:06.236 INFO:teuthology.orchestra.run.smithi133.stdout:Total 11 MB/s | 2.2 GB 03:25 2024-02-21T23:19:07.429 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-02-21T23:19:07.732 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-02-21T23:19:07.733 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-02-21T23:19:21.598 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-02-21T23:19:21.598 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-02-21T23:19:40.547 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-02-21T23:19:43.108 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/204 2024-02-21T23:19:43.196 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_6 2/204 2024-02-21T23:19:43.240 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/204 2024-02-21T23:19:43.298 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/204 2024-02-21T23:19:43.339 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/204 2024-02-21T23:19:43.365 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 6/204 2024-02-21T23:19:43.409 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 7/204 2024-02-21T23:19:43.734 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 8/204 2024-02-21T23:19:43.782 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.0.0-1643.g3bfc837d.el9.x86_64 9/204 2024-02-21T23:19:43.904 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.0.0-1643.g3bfc837d.el9.x86_64 9/204 2024-02-21T23:19:43.933 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.0.0-1643.g3bfc837d.el9.x86_64 10/204 2024-02-21T23:19:44.008 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.0.0-1643.g3bfc837d.el9.x86_64 10/204 2024-02-21T23:19:44.040 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.0.0-1643.g3bfc837d.el9.x86_64 11/204 2024-02-21T23:19:44.133 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/204 2024-02-21T23:19:44.236 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 13/204 2024-02-21T23:19:44.417 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/204 2024-02-21T23:19:44.462 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/204 2024-02-21T23:19:44.490 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.0.0-1643.g3bfc837d.el9.x86_64 16/204 2024-02-21T23:19:44.594 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1643.g3bfc837d.el9.x86_64 16/204 2024-02-21T23:19:44.616 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.0.0-1643.g3bfc837d.el9.x86 17/204 2024-02-21T23:19:44.662 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1643.g3bfc837d.el9.x86 17/204 2024-02-21T23:19:44.684 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 18/204 2024-02-21T23:19:44.708 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 18/204 2024-02-21T23:19:44.881 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 19/204 2024-02-21T23:19:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 19/204 2024-02-21T23:19:45.052 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/204 2024-02-21T23:19:45.157 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/204 2024-02-21T23:19:45.217 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/204 2024-02-21T23:19:45.321 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/204 2024-02-21T23:19:45.368 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 22/204 2024-02-21T23:19:45.582 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/204 2024-02-21T23:19:45.693 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-3.el9.x86_64 23/204 2024-02-21T23:19:45.793 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 24/204 2024-02-21T23:19:45.832 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 25/204 2024-02-21T23:19:45.866 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 26/204 2024-02-21T23:19:45.931 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 27/204 2024-02-21T23:19:45.965 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 28/204 2024-02-21T23:19:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 29/204 2024-02-21T23:19:46.050 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 30/204 2024-02-21T23:19:46.078 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 31/204 2024-02-21T23:19:46.159 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/204 2024-02-21T23:19:46.208 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 33/204 2024-02-21T23:19:46.239 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 34/204 2024-02-21T23:19:46.274 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 35/204 2024-02-21T23:19:46.348 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 36/204 2024-02-21T23:19:46.385 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.0.0-1643.g3bfc837d.el9. 37/204 2024-02-21T23:19:46.427 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.0.0-1643.g3bfc837d.el 38/204 2024-02-21T23:19:46.457 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.0.0-1643.g3bfc837d.el9.x86_6 39/204 2024-02-21T23:19:46.526 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 40/204 2024-02-21T23:19:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-1.el9.noarch 41/204 2024-02-21T23:19:46.654 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 42/204 2024-02-21T23:19:46.787 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 43/204 2024-02-21T23:19:46.840 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 44/204 2024-02-21T23:19:47.354 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 45/204 2024-02-21T23:19:47.427 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 46/204 2024-02-21T23:19:47.455 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 47/204 2024-02-21T23:19:47.486 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 48/204 2024-02-21T23:19:47.534 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 49/204 2024-02-21T23:19:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.0.0-1643.g3bfc837d.el9.x86_6 50/204 2024-02-21T23:19:47.613 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 51/204 2024-02-21T23:19:47.755 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 52/204 2024-02-21T23:19:47.799 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 53/204 2024-02-21T23:19:47.855 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 54/204 2024-02-21T23:19:47.927 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 55/204 2024-02-21T23:19:47.959 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 56/204 2024-02-21T23:19:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 57/204 2024-02-21T23:19:48.042 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 58/204 2024-02-21T23:19:48.237 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 59/204 2024-02-21T23:19:48.433 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 60/204 2024-02-21T23:19:48.483 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 61/204 2024-02-21T23:19:48.616 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 62/204 2024-02-21T23:19:48.654 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 63/204 2024-02-21T23:19:48.684 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 64/204 2024-02-21T23:19:48.726 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 65/204 2024-02-21T23:19:48.819 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 66/204 2024-02-21T23:19:48.859 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 67/204 2024-02-21T23:19:48.907 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 68/204 2024-02-21T23:19:48.942 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 69/204 2024-02-21T23:19:48.966 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 70/204 2024-02-21T23:19:49.044 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 71/204 2024-02-21T23:19:49.182 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 72/204 2024-02-21T23:19:49.215 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 73/204 2024-02-21T23:19:49.243 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 74/204 2024-02-21T23:19:49.281 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 75/204 2024-02-21T23:19:49.316 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-55.el9.x86_64 76/204 2024-02-21T23:19:49.348 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 77/204 2024-02-21T23:19:49.386 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 78/204 2024-02-21T23:19:49.420 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 79/204 2024-02-21T23:19:49.454 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 80/204 2024-02-21T23:19:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 81/204 2024-02-21T23:19:49.530 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 82/204 2024-02-21T23:19:49.615 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 83/204 2024-02-21T23:19:49.661 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 84/204 2024-02-21T23:19:49.692 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 85/204 2024-02-21T23:19:49.738 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 86/204 2024-02-21T23:19:49.861 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 87/204 2024-02-21T23:19:49.934 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 88/204 2024-02-21T23:19:50.653 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 89/204 2024-02-21T23:19:50.698 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 90/204 2024-02-21T23:19:50.732 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 91/204 2024-02-21T23:19:50.830 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 92/204 2024-02-21T23:19:50.883 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 93/204 2024-02-21T23:19:50.960 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 94/204 2024-02-21T23:19:51.001 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 95/204 2024-02-21T23:19:51.187 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/204 2024-02-21T23:19:51.215 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/204 2024-02-21T23:19:51.292 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/204 2024-02-21T23:19:51.927 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/204 2024-02-21T23:19:52.046 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/204 2024-02-21T23:19:53.271 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/204 2024-02-21T23:19:53.352 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/204 2024-02-21T23:19:53.380 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 103/204 2024-02-21T23:19:53.413 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 104/204 2024-02-21T23:19:53.724 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/204 2024-02-21T23:19:53.751 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.0.0-1643.g3bfc837d.el9.x86_64 106/204 2024-02-21T23:19:53.841 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.0.0-1643.g3bfc837d.el9.x86_64 106/204 2024-02-21T23:19:53.872 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 107/204 2024-02-21T23:19:53.909 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/204 2024-02-21T23:19:54.359 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 109/204 2024-02-21T23:19:54.387 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.0.0-1643.g3bfc837d.el9.x86_64 110/204 2024-02-21T23:19:54.458 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.0.0-1643.g3bfc837d.el9.x86_64 110/204 2024-02-21T23:19:54.481 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 111/204 2024-02-21T23:19:56.672 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 112/204 2024-02-21T23:19:56.702 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 112/204 2024-02-21T23:19:57.266 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 112/204 2024-02-21T23:19:57.305 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.0.0-1643.g3bfc837d.el9.x86_64 113/204 2024-02-21T23:19:57.380 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.0.0-1643.g3bfc837d.el9.x86_64 113/204 2024-02-21T23:19:57.380 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-21T23:19:57.380 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-21T23:19:57.380 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:19:57.415 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.0.0-1643.g3bfc837d.el9.x86_64 114/204 2024-02-21T23:20:09.622 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1643.g3bfc837d.el9.x86_64 114/204 2024-02-21T23:20:09.623 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-02-21T23:20:09.623 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-02-21T23:20:09.623 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-02-21T23:20:09.623 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-02-21T23:20:09.623 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-02-21T23:20:09.623 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-02-21T23:20:09.623 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-02-21T23:20:09.623 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:09.784 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.0.0-1643.g3bfc837d.el9.noarch 115/204 2024-02-21T23:20:09.802 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.0.0-1643.g3bfc837d.el9.noarch 115/204 2024-02-21T23:20:09.802 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:20:09.802 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-21T23:20:09.802 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:11.287 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.0.0-1643.g3bfc837d.el9.x86_64 116/204 2024-02-21T23:20:11.313 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.0.0-1643.g3bfc837d.el9.x86_64 116/204 2024-02-21T23:20:11.313 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:20:11.313 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-21T23:20:11.313 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-21T23:20:11.313 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-21T23:20:11.314 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:11.529 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.0.0-1643.g3bfc837d.el9.x86_64 117/204 2024-02-21T23:20:11.551 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.0.0-1643.g3bfc837d.el9.x86_64 117/204 2024-02-21T23:20:11.551 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:20:11.551 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-21T23:20:11.551 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-21T23:20:11.552 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-21T23:20:11.552 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:11.996 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.0.0-1643.g3bfc837d.el9.x86_64 118/204 2024-02-21T23:20:12.019 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.0.0-1643.g3bfc837d.el9.x86_64 118/204 2024-02-21T23:20:12.019 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:20:12.019 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-21T23:20:12.019 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-21T23:20:12.019 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-21T23:20:12.020 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:12.126 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.2-8.el9.noarch 119/204 2024-02-21T23:20:12.174 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 120/204 2024-02-21T23:20:12.230 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 121/204 2024-02-21T23:20:12.414 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 122/204 2024-02-21T23:20:12.463 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 123/204 2024-02-21T23:20:12.540 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/204 2024-02-21T23:20:12.637 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 125/204 2024-02-21T23:20:12.777 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 126/204 2024-02-21T23:20:12.813 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 127/204 2024-02-21T23:20:12.846 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 128/204 2024-02-21T23:20:12.879 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 129/204 2024-02-21T23:20:12.922 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : jq-1.6-16.el9.x86_64 130/204 2024-02-21T23:20:12.966 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-15.el9.x86_64 131/204 2024-02-21T23:20:13.351 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 132/204 2024-02-21T23:20:13.381 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 132/204 2024-02-21T23:20:13.413 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1643.g3bfc837d.e 133/204 2024-02-21T23:20:13.473 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1643.g3bfc837d. 134/204 2024-02-21T23:20:13.505 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1643.g3bfc837d.el9.noa 135/204 2024-02-21T23:20:13.732 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1643.g3bfc837d.el9.noa 135/204 2024-02-21T23:20:13.764 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1643.g3bfc837d.el9.n 136/204 2024-02-21T23:20:14.726 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1643.g3bfc837d.el9.n 136/204 2024-02-21T23:20:14.756 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1643.g3bf 137/204 2024-02-21T23:20:14.855 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1643.g3bf 137/204 2024-02-21T23:20:15.018 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1643.g3bfc837d.el 138/204 2024-02-21T23:20:15.050 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.0.0-1643.g3bfc837d.el9.x86_64 139/204 2024-02-21T23:20:15.073 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1643.g3bfc837d.el9.x86_64 139/204 2024-02-21T23:20:15.073 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:20:15.073 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-21T23:20:15.073 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-21T23:20:15.073 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-21T23:20:15.073 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:15.119 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.0.0-1643.g3bfc837d.el9.noarch 140/204 2024-02-21T23:20:15.127 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1643.g3bfc837d.el9.noarch 140/204 2024-02-21T23:20:15.225 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.0.0-1643.g3bfc837d.el9.x86_64 141/204 2024-02-21T23:20:19.644 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 142/204 2024-02-21T23:20:19.717 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.0.0-1643.g3bfc837d.el9.x86_64 143/204 2024-02-21T23:20:20.575 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-45.el9.x86_64 144/204 2024-02-21T23:20:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 145/204 2024-02-21T23:20:20.637 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 145/204 2024-02-21T23:20:20.638 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:20:20.638 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-21T23:20:20.638 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-21T23:20:20.638 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-21T23:20:20.638 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:20.691 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1643.g3bfc8 146/204 2024-02-21T23:20:20.714 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1643.g3bfc8 146/204 2024-02-21T23:20:20.714 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:20:20.714 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-21T23:20:20.714 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:20.758 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 147/204 2024-02-21T23:20:20.780 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 147/204 2024-02-21T23:20:20.780 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:20:20.780 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-21T23:20:20.780 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-21T23:20:20.780 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-21T23:20:20.781 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:21.084 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 148/204 2024-02-21T23:20:21.106 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 148/204 2024-02-21T23:20:21.106 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-21T23:20:21.106 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-21T23:20:21.106 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-21T23:20:21.106 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-21T23:20:21.106 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:20:21.174 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 149/204 2024-02-21T23:20:21.215 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 150/204 2024-02-21T23:20:21.248 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 151/204 2024-02-21T23:20:21.283 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 152/204 2024-02-21T23:20:21.475 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-shell-2:19.0.0-1643.g3bfc837d.el9.x86_64 153/204 2024-02-21T23:20:21.521 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 154/204 2024-02-21T23:20:21.554 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.0.0-1643.g3bfc837d.el9.x86_ 155/204 2024-02-21T23:20:21.586 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 156/204 2024-02-21T23:20:21.616 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 157/204 2024-02-21T23:20:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 158/204 2024-02-21T23:20:21.753 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 159/204 2024-02-21T23:20:21.789 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 160/204 2024-02-21T23:20:22.458 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 161/204 2024-02-21T23:20:22.495 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 161/204 2024-02-21T23:20:22.543 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 161/204 2024-02-21T23:20:22.593 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/204 2024-02-21T23:20:29.698 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:19.0.0-1643.g3bfc837d.el9.noarch 163/204 2024-02-21T23:21:11.516 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 164/204 2024-02-21T23:21:12.136 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-debuginfo-2:19.0.0-1643.g3bfc837d.el 165/204 2024-02-21T23:21:12.309 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 166/204 2024-02-21T23:21:17.120 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.0.0-1 167/204 2024-02-21T23:21:19.203 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 168/204 2024-02-21T23:21:29.087 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 169/204 2024-02-21T23:21:57.078 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 170/204 2024-02-21T23:22:14.309 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 171/204 2024-02-21T23:23:21.529 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-debuginfo-2:19.0.0-1643.g3bfc837d.e 172/204 2024-02-21T23:23:22.313 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 173/204 2024-02-21T23:23:22.351 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-debuginfo-2:19.0.0-1643.g3bfc837d.el9 174/204 2024-02-21T23:23:27.108 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-debuginfo-2:19.0.0-1643.g3bfc837d 175/204 2024-02-21T23:23:32.924 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados2-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 176/204 2024-02-21T23:23:33.013 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librbd1-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x8 177/204 2024-02-21T23:23:33.137 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-debuginfo-2:19.0.0-1643.g3bfc837d 178/204 2024-02-21T23:23:33.264 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-debuginfo-2:19.0.0-1643.g3bfc837d. 179/204 2024-02-21T23:23:33.330 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-debuginfo-2:19.0.0-1643.g3bfc837d.el 180/204 2024-02-21T23:23:33.419 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-debuginfo-2:19.0.0-1643.g3bfc837d.el 181/204 2024-02-21T23:23:38.959 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 182/204 2024-02-21T23:23:39.119 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-debuginfo-2:19.0.0-1643.g3bfc837d.el9 183/204 2024-02-21T23:23:39.943 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x8 184/204 2024-02-21T23:23:40.062 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 185/204 2024-02-21T23:23:40.150 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.4-9.el9.x86_64 186/204 2024-02-21T23:23:40.192 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.7.4-5.el9.x86_64 187/204 2024-02-21T23:23:40.226 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 188/204 2024-02-21T23:23:40.251 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 189/204 2024-02-21T23:23:40.276 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 190/204 2024-02-21T23:23:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 191/204 2024-02-21T23:23:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 192/204 2024-02-21T23:23:40.407 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 192/204 2024-02-21T23:23:40.451 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 193/204 2024-02-21T23:23:40.479 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 194/204 2024-02-21T23:23:40.505 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 195/204 2024-02-21T23:23:40.530 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 196/204 2024-02-21T23:23:40.530 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 197/204 2024-02-21T23:23:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 197/204 2024-02-21T23:23:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 198/204 2024-02-21T23:23:40.637 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 199/204 2024-02-21T23:23:40.638 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 199/204 2024-02-21T23:23:40.817 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 199/204 2024-02-21T23:23:40.817 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 200/204 2024-02-21T23:23:40.909 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 200/204 2024-02-21T23:23:40.933 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 201/204 2024-02-21T23:23:40.969 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 202/204 2024-02-21T23:23:41.001 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 203/204 2024-02-21T23:23:41.068 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 204/204 2024-02-21T23:23:50.467 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 204/204 2024-02-21T23:23:50.467 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.0.0-1643.g3bfc837d.el9.x86_64 1/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.0.0-1643.g3bfc837d.el9.x86_64 2/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 3/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 4/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-debuginfo-2:19.0.0-1643.g3bfc837d.el 5/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_6 6/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 7/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 8/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1643.g3bfc8 9/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.0.0-1 10/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.0.0-1643.g3bfc837d.el9.x86_64 11/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 12/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.0.0-1643.g3bfc837d.el9.x86_64 13/204 2024-02-21T23:23:50.468 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 14/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.0.0-1643.g3bfc837d.el9.x86_64 15/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 16/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.0.0-1643.g3bfc837d.el9.x86_64 17/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 18/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 19/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-debuginfo-2:19.0.0-1643.g3bfc837d.e 20/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.0.0-1643.g3bfc837d.el9.x86_64 21/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.0.0-1643.g3bfc837d.el9.x86_64 22/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 23/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 24/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-shell-2:19.0.0-1643.g3bfc837d.el9.x86_64 25/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.0.0-1643.g3bfc837d.el9.x86_ 26/204 2024-02-21T23:23:50.469 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.0.0-1643.g3bfc837d.el9.x86_64 27/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-debuginfo-2:19.0.0-1643.g3bfc837d.el9 28/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.0.0-1643.g3bfc837d.el9.x86_64 29/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.0.0-1643.g3bfc837d.el9.x86_6 30/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-debuginfo-2:19.0.0-1643.g3bfc837d 31/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-debuginfo-2:19.0.0-1643.g3bfc837d.el9. 32/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.0.0-1643.g3bfc837d.el9.x86 33/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x8 34/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.0.0-1643.g3bfc837d.el9.x86_64 35/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1643.g3bfc837d.el 36/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.0.0-1643.g3bfc837d.el9. 37/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.0.0-1643.g3bfc837d.el9.x86_6 38/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-debuginfo-2:19.0.0-1643.g3bfc837d 39/204 2024-02-21T23:23:50.470 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.0.0-1643.g3bfc837d.el9.x86_64 40/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-debuginfo-2:19.0.0-1643.g3bfc837d. 41/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 42/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-debuginfo-2:19.0.0-1643.g3bfc837d.el 43/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 44/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-debuginfo-2:19.0.0-1643.g3bfc837d.el 45/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 46/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x 47/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 48/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-debuginfo-2:19.0.0-1643.g3bfc837d.el9 49/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 50/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x8 51/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1643.g3bfc837d. 52/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1643.g3bfc837d.el9.noa 53/204 2024-02-21T23:23:50.471 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1643.g3bfc837d.el9.n 54/204 2024-02-21T23:23:50.472 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1643.g3bf 55/204 2024-02-21T23:23:50.472 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1643.g3bfc837d.el 56/204 2024-02-21T23:23:50.472 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1643.g3bfc837d.el9.noarch 57/204 2024-02-21T23:23:50.472 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1643.g3bfc837d.e 58/204 2024-02-21T23:23:50.472 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.0.0-1643.g3bfc837d.el9.noarch 59/204 2024-02-21T23:23:50.472 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 60/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:19.0.0-1643.g3bfc837d.el9.noarch 61/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 62/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 63/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : jq-1.6-16.el9.x86_64 64/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 65/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 66/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 67/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 68/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 69/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 70/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 71/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 72/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 73/204 2024-02-21T23:23:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.2-8.el9.noarch 74/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.7.4-5.el9.x86_64 75/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 76/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 77/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.4-9.el9.x86_64 78/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 79/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 80/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 82/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 83/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 84/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 85/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 86/204 2024-02-21T23:23:50.474 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 87/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 88/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 89/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 90/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 91/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 92/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-13.11-1.el9.x86_64 93/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 94/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 95/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 96/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 97/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 98/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 99/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 100/204 2024-02-21T23:23:50.475 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 101/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 102/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 103/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 104/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 105/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 106/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 107/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 108/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 109/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 110/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 111/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 112/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 113/204 2024-02-21T23:23:50.476 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 114/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 115/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 116/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 117/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 118/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 119/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 120/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 121/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 122/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 123/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 124/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 125/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 126/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 127/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 128/204 2024-02-21T23:23:50.477 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 129/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 130/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 131/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 132/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 133/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 134/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 135/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 136/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 137/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 138/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 139/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 140/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 141/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 142/204 2024-02-21T23:23:50.478 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 143/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 144/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 145/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 146/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 147/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 148/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 149/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 150/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 151/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 152/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 153/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 154/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 155/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 156/204 2024-02-21T23:23:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 157/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 158/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 159/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 160/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 161/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 162/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 163/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 164/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 165/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 166/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 167/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 168/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 169/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 170/204 2024-02-21T23:23:50.480 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-3.el9.x86_64 171/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 172/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.0.0-1643.g3bfc837d.el9.x86_64 173/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 174/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.0.0-1643.g3bfc837d.el9.x86_64 175/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 176/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 177/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 178/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 179/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 180/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 181/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 182/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 183/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 184/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : acl-2.3.1-4.el9.x86_64 185/204 2024-02-21T23:23:50.481 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : acl-2.3.1-3.el9.x86_64 186/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 187/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 188/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 189/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 190/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 191/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 192/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 193/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 194/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 195/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 196/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 197/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 198/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/204 2024-02-21T23:23:50.482 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/204 2024-02-21T23:23:50.483 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/204 2024-02-21T23:23:50.483 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/204 2024-02-21T23:23:50.483 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 203/204 2024-02-21T23:23:53.507 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 204/204 2024-02-21T23:23:53.507 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:53.507 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-02-21T23:23:53.507 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-21T23:23:53.507 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-21T23:23:53.507 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-21T23:23:53.507 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-21T23:23:53.507 INFO:teuthology.orchestra.run.smithi133.stdout: acl-2.3.1-4.el9.x86_64 2024-02-21T23:23:53.507 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-21T23:23:53.508 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.2-8.el9.noarch 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.7.4-5.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.510 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.511 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:19.0.0-1643.g3bfc837d.el9.noarch 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.4-9.el9.x86_64 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: jq-1.6-16.el9.x86_64 2024-02-21T23:23:53.512 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-21T23:23:53.514 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-13.11-1.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-21T23:23:53.515 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-21T23:23:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-21T23:23:53.517 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-21T23:23:53.518 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-21T23:23:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-21T23:23:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-debuginfo-2:19.0.0-1643.g3bfc837d.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-3.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-21T23:23:53.521 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-21T23:23:53.522 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-21T23:23:53.522 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-21T23:23:53.522 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-21T23:23:53.522 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-21T23:23:53.522 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:53.522 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-02-21T23:23:54.235 DEBUG:teuthology.parallel:result is None 2024-02-21T23:23:54.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3bfc837d7b45ce8e095d92fa9c342964cea5e95f 2024-02-21T23:23:54.380 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-21T23:23:54.403 INFO:teuthology.orchestra.run.smithi133.stdout:19.0.0-1643.g3bfc837d.el9 2024-02-21T23:23:54.404 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1643.g3bfc837d.el9 2024-02-21T23:23:54.404 INFO:teuthology.task.install:The correct ceph version 19.0.0-1643.g3bfc837d is installed. 2024-02-21T23:23:54.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3bfc837d7b45ce8e095d92fa9c342964cea5e95f 2024-02-21T23:23:54.550 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-21T23:23:54.580 INFO:teuthology.orchestra.run.smithi142.stdout:19.0.0-1643.g3bfc837d.el9 2024-02-21T23:23:54.580 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1643.g3bfc837d.el9 2024-02-21T23:23:54.580 INFO:teuthology.task.install:The correct ceph version 19.0.0-1643.g3bfc837d is installed. 2024-02-21T23:23:54.582 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-21T23:23:54.582 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:54.582 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-21T23:23:54.612 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:23:54.612 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-21T23:23:54.654 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-21T23:23:54.655 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:54.655 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-02-21T23:23:54.682 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-21T23:23:54.749 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:23:54.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-02-21T23:23:54.779 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-21T23:23:54.849 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-21T23:23:54.849 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:54.849 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-21T23:23:54.875 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-21T23:23:54.942 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:23:54.943 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-21T23:23:54.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-21T23:23:55.041 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-21T23:23:55.041 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:55.041 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-02-21T23:23:55.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-21T23:23:55.134 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:23:55.134 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-02-21T23:23:55.163 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-21T23:23:55.232 INFO:teuthology.run_tasks:Running task ceph... 2024-02-21T23:23:55.322 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-02-21T23:23:55.322 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2024-02-21T23:23:55.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /var/log/ceph 2024-02-21T23:23:55.352 INFO:tasks.ceph:Disabling ceph logrotate... 2024-02-21T23:23:55.352 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-21T23:23:55.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-21T23:23:55.420 INFO:tasks.ceph:Creating extra log directories... 2024-02-21T23:23:55.420 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-21T23:23:55.460 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-21T23:23:55.495 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-02-21T23:23:55.495 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-02-21T23:23:55.495 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-02-21T23:23:55.496 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:55.496 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-02-21T23:23:55.540 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-02-21T23:23:55.658 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-02-21T23:23:55.686 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:23:55.686 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-02-21T23:23:55.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-02-21T23:23:55.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-02-21T23:23:55.849 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-02-21T23:23:55.850 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-02-21T23:23:55.850 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021', 'branch': 'wip-lusov-qdb', 'description': 'fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7570021', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-qdb'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}, 'watchdog': {'daemon_restart': 'normal'}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'ceph-fuse': {'client.0': {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-qdb', 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 6562, 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f', 'sleep_before_teardown': 0, 'subset': '111/10000', 'suite': 'fs', 'suite_branch': 'wip-lusov-qdb', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi133.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIQO4F2tzpA2TGhD0+Q5J562sN4OU5mir5R/JLR2UmtSabM3oZhxjURS0e0nm69nVx0/HmIPnB93HuV8bXkZtqM=', 'smithi142.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIY6+OhPMMUUKiclL98O2Lt81V/BXxVGzuWGo/6K6ZtYxlt+DbvCAGa34duU8PucAHFMXB8TFlHfqV/kqyG528E='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'debuginfo': True, 'flavor': 'default', 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'sha1': '3bfc837d7b45ce8e095d92fa9c342964cea5e95f', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}, 'watchdog': {'daemon_restart': 'normal'}, 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'workunit': {'clients': {'all': ['suites/dbench.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b1dac5519c57c269ea98a22fb7729016a1d0e4d2', 'timestamp': '2024-02-21_22:31:03', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-02-21T23:23:55.850 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-21T23:23:55.853 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-21T23:23:55.899 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2024-02-21T23:23:55.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/run/ceph 2024-02-21T23:23:55.966 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:55.967 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-02-21T23:23:55.990 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-21T23:23:55.990 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-02-21T23:23:56.044 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-21T23:23:56.044 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-21T23:23:56.044 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-21T23:23:56.044 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-21T23:23:56.044 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-02-21T23:23:56.044 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-02-21 23:23:41.422792948 +0000 2024-02-21T23:23:56.045 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-02-21 23:08:59.845556897 +0000 2024-02-21T23:23:56.045 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-02-21 23:08:59.845556897 +0000 2024-02-21T23:23:56.045 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-02-21 23:08:59.845556897 +0000 2024-02-21T23:23:56.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-21T23:23:56.110 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-02-21T23:23:56.110 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-02-21T23:23:56.110 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000205309 s, 2.5 MB/s 2024-02-21T23:23:56.111 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-21T23:23:56.167 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-02-21T23:23:56.222 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-21T23:23:56.223 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-21T23:23:56.223 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-21T23:23:56.223 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-21T23:23:56.223 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-02-21T23:23:56.223 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-02-21 23:23:41.422792948 +0000 2024-02-21T23:23:56.223 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-02-21 23:09:00.194550176 +0000 2024-02-21T23:23:56.223 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-02-21 23:09:00.194550176 +0000 2024-02-21T23:23:56.223 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-02-21 23:09:00.194550176 +0000 2024-02-21T23:23:56.223 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-21T23:23:56.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-02-21T23:23:56.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-02-21T23:23:56.288 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000240339 s, 2.1 MB/s 2024-02-21T23:23:56.289 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-21T23:23:56.345 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-02-21T23:23:56.402 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-21T23:23:56.402 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-21T23:23:56.402 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-21T23:23:56.402 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-21T23:23:56.402 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-02-21T23:23:56.402 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-02-21 23:23:41.423792928 +0000 2024-02-21T23:23:56.402 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-02-21 23:09:00.537543570 +0000 2024-02-21T23:23:56.402 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-02-21 23:09:00.537543570 +0000 2024-02-21T23:23:56.402 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-02-21 23:09:00.537543570 +0000 2024-02-21T23:23:56.403 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-21T23:23:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-02-21T23:23:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-02-21T23:23:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000206658 s, 2.5 MB/s 2024-02-21T23:23:56.468 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-21T23:23:56.524 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-02-21T23:23:56.579 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-21T23:23:56.579 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-21T23:23:56.579 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-21T23:23:56.579 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-21T23:23:56.579 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-02-21T23:23:56.579 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-02-21 23:23:41.423792928 +0000 2024-02-21T23:23:56.580 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-02-21 23:09:00.890536771 +0000 2024-02-21T23:23:56.580 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-02-21 23:09:00.890536771 +0000 2024-02-21T23:23:56.580 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-02-21 23:09:00.890536771 +0000 2024-02-21T23:23:56.580 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-21T23:23:56.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-02-21T23:23:56.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-02-21T23:23:56.649 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000208363 s, 2.5 MB/s 2024-02-21T23:23:56.650 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-21T23:23:56.706 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-21T23:23:56.706 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:23:56.706 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-02-21T23:23:56.723 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-21T23:23:56.723 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-02-21T23:23:56.780 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-21T23:23:56.781 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-21T23:23:56.781 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-02-21T23:23:56.781 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-21T23:23:56.781 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-21T23:23:56.781 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-21 23:17:37.625089119 +0000 2024-02-21T23:23:56.781 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-21 23:09:19.971063430 +0000 2024-02-21T23:23:56.782 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-21 23:09:19.971063430 +0000 2024-02-21T23:23:56.782 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-21 23:09:19.971063430 +0000 2024-02-21T23:23:56.782 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-21T23:23:56.849 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-21T23:23:56.849 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-21T23:23:56.849 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000272395 s, 1.9 MB/s 2024-02-21T23:23:56.850 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-21T23:23:56.907 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-02-21T23:23:56.964 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-21T23:23:56.965 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-21T23:23:56.965 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-21T23:23:56.965 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-21T23:23:56.965 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-21T23:23:56.965 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-21 23:17:37.625089119 +0000 2024-02-21T23:23:56.965 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-21 23:09:20.305057195 +0000 2024-02-21T23:23:56.965 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-21 23:09:20.305057195 +0000 2024-02-21T23:23:56.965 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-21 23:09:20.305057195 +0000 2024-02-21T23:23:56.965 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-21T23:23:57.032 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-21T23:23:57.032 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-21T23:23:57.032 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000194534 s, 2.6 MB/s 2024-02-21T23:23:57.032 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-21T23:23:57.088 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-02-21T23:23:57.143 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-21T23:23:57.143 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-21T23:23:57.143 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-21T23:23:57.143 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-21T23:23:57.143 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-21T23:23:57.143 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-21 23:17:37.626089101 +0000 2024-02-21T23:23:57.144 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-21 23:09:20.658050605 +0000 2024-02-21T23:23:57.144 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-21 23:09:20.658050605 +0000 2024-02-21T23:23:57.144 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-21 23:09:20.658050605 +0000 2024-02-21T23:23:57.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-21T23:23:57.209 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-21T23:23:57.209 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-21T23:23:57.209 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000203143 s, 2.5 MB/s 2024-02-21T23:23:57.210 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-21T23:23:57.267 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-02-21T23:23:57.323 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-21T23:23:57.323 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-21T23:23:57.323 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-21T23:23:57.323 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-21T23:23:57.323 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-21T23:23:57.323 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-21 23:17:37.626089101 +0000 2024-02-21T23:23:57.323 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-21 23:09:21.008044070 +0000 2024-02-21T23:23:57.323 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-21 23:09:21.008044070 +0000 2024-02-21T23:23:57.323 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-21 23:09:21.008044070 +0000 2024-02-21T23:23:57.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-21T23:23:57.389 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-21T23:23:57.389 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-21T23:23:57.389 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000205722 s, 2.5 MB/s 2024-02-21T23:23:57.390 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-21T23:23:57.447 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-21T23:23:57.447 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-02-21T23:23:57.447 INFO:tasks.ceph:Generating config... 2024-02-21T23:23:57.448 INFO:tasks.ceph:[client] client mount timeout = 600 2024-02-21T23:23:57.448 INFO:tasks.ceph:[client] debug client = 20 2024-02-21T23:23:57.448 INFO:tasks.ceph:[client] debug ms = 1 2024-02-21T23:23:57.448 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-02-21T23:23:57.448 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-02-21T23:23:57.449 INFO:tasks.ceph:[global] mds beacon grace = 60 2024-02-21T23:23:57.449 INFO:tasks.ceph:[global] mds heartbeat grace = 60 2024-02-21T23:23:57.449 INFO:tasks.ceph:[global] osd heartbeat grace = 60 2024-02-21T23:23:57.449 INFO:tasks.ceph:[mds] debug mds = 20 2024-02-21T23:23:57.449 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-02-21T23:23:57.449 INFO:tasks.ceph:[mds] debug ms = 1 2024-02-21T23:23:57.449 INFO:tasks.ceph:[mds] mds debug frag = True 2024-02-21T23:23:57.449 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-02-21T23:23:57.449 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-02-21T23:23:57.449 INFO:tasks.ceph:[mds] mds valgrind exit = True 2024-02-21T23:23:57.449 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mgr] debug ms = 1 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mon] debug mon = 20 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mon] debug ms = 1 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mon] debug paxos = 20 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-02-21T23:23:57.450 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-02-21T23:23:57.450 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-02-21T23:23:57.450 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-02-21T23:23:57.450 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-02-21T23:23:57.450 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-02-21T23:23:57.450 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] debug ms = 1 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] debug osd = 20 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-02-21T23:23:57.451 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-02-21T23:23:57.452 INFO:tasks.ceph:Setting up mon.a... 2024-02-21T23:23:57.452 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-02-21T23:23:57.507 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.keyring 2024-02-21T23:23:57.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-02-21T23:23:57.557 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-21T23:23:57.584 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.133'), ('mon.b', '172.21.15.142'), ('mon.c', '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]')] 2024-02-21T23:23:57.584 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.133,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]', 'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-02-21T23:23:57.585 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:57.585 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-02-21T23:23:57.640 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.133 --add b 172.21.15.142 --addv c '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:23:57.724 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: generated fsid 1f1c3b6d-6263-48c3-85b7-ccc808770c4d 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = quincy 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 0 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 1f1c3b6d-6263-48c3-85b7-ccc808770c4d 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:last_changed 2024-02-21T23:23:57.739165+0000 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-02-21T23:23:57.739165+0000 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:min_mon_release 17 (quincy) 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:election_strategy: 1 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:2: [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] mon.c 2024-02-21T23:23:57.725 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-02-21T23:23:57.726 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-02-21T23:23:57.741 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1f1c3b6d-6263-48c3-85b7-ccc808770c4d... 2024-02-21T23:23:57.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-21T23:23:57.784 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-02-21T23:23:57.832 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-02-21T23:23:57.833 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-02-21T23:23:57.834 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.133,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: mds beacon grace = 60 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: mds heartbeat grace = 60 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: osd heartbeat grace = 60 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 1f1c3b6d-6263-48c3-85b7-ccc808770c4d 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-02-21T23:23:57.835 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-02-21T23:23:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore compression mode = aggressive 2024-02-21T23:23:57.837 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 20 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 20 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 10 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: osd fast shutdown = False 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-02-21T23:23:57.838 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-02-21T23:23:57.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-02-21T23:23:57.840 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd crush smoke test = False 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-21T23:23:57.841 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.e] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.f] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-02-21T23:23:57.842 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = True 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: mds valgrind exit = True 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = True 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-02-21T23:23:57.843 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.844 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.845 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-02-21T23:23:57.846 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.133,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: mds beacon grace = 60 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: mds heartbeat grace = 60 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: osd heartbeat grace = 60 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 1f1c3b6d-6263-48c3-85b7-ccc808770c4d 2024-02-21T23:23:57.847 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-02-21T23:23:57.848 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore compression mode = aggressive 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 20 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 20 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 10 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-02-21T23:23:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: osd fast shutdown = False 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-02-21T23:23:57.851 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-02-21T23:23:57.852 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd crush smoke test = False 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-02-21T23:23:57.853 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-02-21T23:23:57.854 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.e] 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.f] 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: mds valgrind exit = True 2024-02-21T23:23:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-02-21T23:23:57.856 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-02-21T23:23:57.856 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-02-21T23:23:57.856 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-02-21T23:23:57.857 INFO:tasks.ceph:Creating admin key on mon.a... 2024-02-21T23:23:57.858 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-02-21T23:23:57.954 INFO:tasks.ceph:Copying monmap to all nodes... 2024-02-21T23:23:57.954 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:57.954 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-02-21T23:23:57.969 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:57.969 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-02-21T23:23:58.024 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2024-02-21T23:23:58.025 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:58.025 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-21T23:23:58.025 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-21T23:23:58.104 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:23:58.104 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:23:58.160 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi142.front.sepia.ceph.com 2024-02-21T23:23:58.160 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:23:58.160 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-21T23:23:58.161 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-21T23:23:58.201 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:23:58.201 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:23:58.257 INFO:tasks.ceph:Setting up mon nodes... 2024-02-21T23:23:58.257 INFO:tasks.ceph:Setting up mgr nodes... 2024-02-21T23:23:58.258 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-02-21T23:23:58.314 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-02-21T23:23:58.317 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-02-21T23:23:58.389 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-02-21T23:23:58.391 INFO:tasks.ceph:Setting up mds nodes... 2024-02-21T23:23:58.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-02-21T23:23:58.446 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-02-21T23:23:58.448 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-02-21T23:23:58.475 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-02-21T23:23:58.570 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-02-21T23:23:58.573 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-02-21T23:23:58.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-02-21T23:23:58.694 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-02-21T23:23:58.696 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-02-21T23:23:58.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-02-21T23:23:58.779 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-02-21T23:23:58.781 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-02-21T23:23:58.808 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-02-21T23:23:58.903 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-02-21T23:23:58.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-02-21T23:23:58.932 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-02-21T23:23:59.028 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-02-21T23:23:59.031 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-02-21T23:23:59.057 INFO:tasks.ceph_client:Setting up client nodes... 2024-02-21T23:23:59.057 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-21T23:23:59.101 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-02-21T23:23:59.117 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-02-21T23:23:59.118 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-02-21T23:23:59.118 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-02-21T23:23:59.145 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-21T23:23:59.145 INFO:tasks.ceph:role: osd.0 2024-02-21T23:23:59.145 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com 2024-02-21T23:23:59.145 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-21T23:23:59.218 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-21T23:23:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-21T23:23:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-21T23:23:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-21T23:23:59.219 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-21T23:23:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-02-21T23:23:59.219 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-21T23:23:59.219 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-21T23:23:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-21T23:23:59.219 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-21T23:23:59.713 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-02-21T23:23:59.714 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-02-21T23:23:59.715 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-02-21T23:23:59.819 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-02-21T23:23:59.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-02-21T23:23:59.922 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-21T23:23:59.922 INFO:tasks.ceph:role: osd.1 2024-02-21T23:23:59.922 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com 2024-02-21T23:23:59.922 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-21T23:23:59.996 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-21T23:23:59.996 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-21T23:23:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-21T23:23:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-21T23:23:59.997 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-21T23:23:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-02-21T23:23:59.997 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-21T23:23:59.997 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-21T23:23:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-21T23:23:59.997 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-21T23:24:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-02-21T23:24:00.473 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-02-21T23:24:00.473 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-02-21T23:24:00.558 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-02-21T23:24:00.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-02-21T23:24:00.658 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-21T23:24:00.658 INFO:tasks.ceph:role: osd.2 2024-02-21T23:24:00.658 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com 2024-02-21T23:24:00.658 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-21T23:24:00.733 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-21T23:24:01.222 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-02-21T23:24:01.224 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-02-21T23:24:01.224 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-02-21T23:24:01.360 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-02-21T23:24:01.396 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-02-21T23:24:01.465 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-21T23:24:01.465 INFO:tasks.ceph:role: osd.3 2024-02-21T23:24:01.465 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com 2024-02-21T23:24:01.466 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-02-21T23:24:01.542 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-02-21T23:24:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-21T23:24:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-21T23:24:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-21T23:24:01.542 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-21T23:24:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-02-21T23:24:01.542 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-21T23:24:01.542 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-21T23:24:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-21T23:24:01.543 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-21T23:24:02.054 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-02-21T23:24:02.056 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-02-21T23:24:02.056 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-02-21T23:24:02.208 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-02-21T23:24:02.245 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:02.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:02.357+0000 7f470f1dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-02-21T23:24:02.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:02.357+0000 7f470f1dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-02-21T23:24:02.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:02.357+0000 7f470f1dd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-02-21T23:24:02.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:02.357+0000 7f470f1dd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-02-21T23:24:02.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:02.357+0000 7f470f1dd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-02-21T23:24:02.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:02.357+0000 7f470f1dd640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-02-21T23:24:05.197 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-21T23:24:05.228 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:05.333+0000 7f3690204640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-02-21T23:24:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:05.333+0000 7f3690204640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-02-21T23:24:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:05.333+0000 7f3690204640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-02-21T23:24:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:05.333+0000 7f3690204640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-02-21T23:24:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:05.333+0000 7f3690204640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-02-21T23:24:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:05.334+0000 7f3690204640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-02-21T23:24:08.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-21T23:24:08.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:08.305+0000 7ff793613640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-02-21T23:24:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:08.305+0000 7ff793613640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-02-21T23:24:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:08.305+0000 7ff793613640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-02-21T23:24:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:08.305+0000 7ff793613640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-02-21T23:24:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:08.305+0000 7ff793613640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-02-21T23:24:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:08.305+0000 7ff793613640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-02-21T23:24:11.142 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-21T23:24:11.173 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:11.280+0000 7f5d88066640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-02-21T23:24:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:11.281+0000 7f5d88066640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-02-21T23:24:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:11.281+0000 7f5d88066640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-02-21T23:24:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:11.281+0000 7f5d88066640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-02-21T23:24:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:11.281+0000 7f5d88066640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-02-21T23:24:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:11.281+0000 7f5d88066640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-02-21T23:24:14.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-21T23:24:14.155 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-02-21T23:24:14.187 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-21T23:24:14.187 INFO:tasks.ceph:role: osd.4 2024-02-21T23:24:14.187 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com 2024-02-21T23:24:14.187 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-21T23:24:14.265 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-21T23:24:14.265 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-21T23:24:14.265 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-21T23:24:14.265 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-21T23:24:14.265 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-21T23:24:14.265 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-02-21T23:24:14.265 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-21T23:24:14.265 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-21T23:24:14.265 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-21T23:24:14.266 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-21T23:24:14.782 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-02-21T23:24:14.784 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-02-21T23:24:14.784 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-02-21T23:24:14.931 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-02-21T23:24:14.969 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-02-21T23:24:15.038 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-21T23:24:15.038 INFO:tasks.ceph:role: osd.5 2024-02-21T23:24:15.039 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com 2024-02-21T23:24:15.039 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-21T23:24:15.114 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-21T23:24:15.114 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-21T23:24:15.114 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-21T23:24:15.114 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-21T23:24:15.115 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-21T23:24:15.115 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-02-21T23:24:15.115 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-21T23:24:15.115 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-21T23:24:15.115 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-21T23:24:15.115 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-21T23:24:15.557 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-02-21T23:24:15.558 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-02-21T23:24:15.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-02-21T23:24:15.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-02-21T23:24:15.697 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-02-21T23:24:15.767 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-21T23:24:15.767 INFO:tasks.ceph:role: osd.6 2024-02-21T23:24:15.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com 2024-02-21T23:24:15.768 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-02-21T23:24:15.843 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-02-21T23:24:15.844 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-21T23:24:15.844 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-21T23:24:15.844 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-21T23:24:15.844 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-21T23:24:15.845 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-02-21T23:24:15.845 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-21T23:24:15.845 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-21T23:24:15.845 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-21T23:24:15.845 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-21T23:24:16.327 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-02-21T23:24:16.330 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-02-21T23:24:16.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-02-21T23:24:16.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-02-21T23:24:16.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-02-21T23:24:16.598 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-21T23:24:16.598 INFO:tasks.ceph:role: osd.7 2024-02-21T23:24:16.598 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com 2024-02-21T23:24:16.599 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-21T23:24:16.675 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-21T23:24:17.157 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-02-21T23:24:17.159 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-02-21T23:24:17.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-02-21T23:24:17.300 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-02-21T23:24:17.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:17.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:17.457+0000 7f0a8c2ae640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-02-21T23:24:17.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:17.457+0000 7f0a8c2ae640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-02-21T23:24:17.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:17.457+0000 7f0a8c2ae640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-02-21T23:24:17.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:17.457+0000 7f0a8c2ae640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-02-21T23:24:17.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:17.457+0000 7f0a8c2ae640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-02-21T23:24:17.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:17.457+0000 7f0a8c2ae640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-02-21T23:24:20.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-21T23:24:20.335 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:20.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:20.439+0000 7ffab9ccc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-02-21T23:24:20.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:20.439+0000 7ffab9ccc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-02-21T23:24:20.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:20.439+0000 7ffab9ccc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-02-21T23:24:20.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:20.439+0000 7ffab9ccc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-02-21T23:24:20.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:20.439+0000 7ffab9ccc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-02-21T23:24:20.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:20.440+0000 7ffab9ccc640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-02-21T23:24:23.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-21T23:24:23.312 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:23.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:23.418+0000 7f9dcf38a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-02-21T23:24:23.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:23.419+0000 7f9dcf38a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-02-21T23:24:23.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:23.419+0000 7f9dcf38a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-02-21T23:24:23.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:23.419+0000 7f9dcf38a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-02-21T23:24:23.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:23.419+0000 7f9dcf38a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-02-21T23:24:23.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:23.419+0000 7f9dcf38a640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-02-21T23:24:25.851 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:24:25.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:24:26.254 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-02-21T23:24:26.285 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:26.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:26.390+0000 7fa388866640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-02-21T23:24:26.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:26.390+0000 7fa388866640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-02-21T23:24:26.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:26.390+0000 7fa388866640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-02-21T23:24:26.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:26.390+0000 7fa388866640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-02-21T23:24:26.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:26.390+0000 7fa388866640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-02-21T23:24:26.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:26.391+0000 7fa388866640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-02-21T23:24:29.230 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-02-21T23:24:29.261 INFO:tasks.ceph:Reading keys from all nodes... 2024-02-21T23:24:29.262 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:29.262 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-02-21T23:24:29.294 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:29.294 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-02-21T23:24:29.364 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:29.364 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-02-21T23:24:29.433 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:29.434 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-02-21T23:24:29.502 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:29.503 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-02-21T23:24:29.571 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:29.572 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-02-21T23:24:29.640 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:29.640 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-02-21T23:24:29.708 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:29.708 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-02-21T23:24:29.776 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:29.776 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-02-21T23:24:29.808 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:29.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-02-21T23:24:29.880 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:29.881 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-02-21T23:24:29.951 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:29.951 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-02-21T23:24:30.022 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:30.022 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-02-21T23:24:30.091 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:30.091 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-02-21T23:24:30.161 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:30.161 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-02-21T23:24:30.230 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:30.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-02-21T23:24:30.298 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:30.298 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-02-21T23:24:30.315 INFO:tasks.ceph:Adding keys to all mons... 2024-02-21T23:24:30.315 DEBUG:teuthology.orchestra.run.smithi133:> sudo tee -a /etc/ceph/ceph.keyring 2024-02-21T23:24:30.358 DEBUG:teuthology.orchestra.run.smithi142:> sudo tee -a /etc/ceph/ceph.keyring 2024-02-21T23:24:30.383 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.y] 2024-02-21T23:24:30.383 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAOhtZlx1CYExAADw1OdhJ+N/XUVRLQ86IeWg== 2024-02-21T23:24:30.383 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-02-21T23:24:30.384 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAOhtZle8Z5GxAAuhj3K1bupxfYneeGexEONA== 2024-02-21T23:24:30.384 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-02-21T23:24:30.384 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAOhtZl+CHfIhAA4MaQg9cDkpRovwp4b6CfTg== 2024-02-21T23:24:30.384 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.e] 2024-02-21T23:24:30.384 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAOhtZlERJDKhAARCUqAHRiQSQttiolEwTOjw== 2024-02-21T23:24:30.385 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2024-02-21T23:24:30.385 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAShtZl3EFcFRAATIHwVh8b6XqW707iZaLHVw== 2024-02-21T23:24:30.385 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2024-02-21T23:24:30.385 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAVhtZl9Sb1ExAAUeLbOPJ8nrm2x98AlOrFNQ== 2024-02-21T23:24:30.385 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2024-02-21T23:24:30.385 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAYhtZl9Yw+EhAAaZ7hcv9qb2vdTjWjXL05vw== 2024-02-21T23:24:30.386 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.3] 2024-02-21T23:24:30.386 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAbhtZltqDBEBAAgBOPm19IcB93dxwYMyDE7w== 2024-02-21T23:24:30.386 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.x] 2024-02-21T23:24:30.386 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAOhtZl9xQRGBAAza2kuugJdrQYjyK++VrWUw== 2024-02-21T23:24:30.386 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAOhtZlC0tQLxAAZYKqsmGF6SQDlu5daIM3ZA== 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAOhtZlTOS2NhAAtE9wDO8wq2T9FbGP6mtr+w== 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.f] 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAPhtZla5WRAhAAHDe9igOGbPzB3SCCS6qN9g== 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.4] 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAhhtZlN9lNGxAAyEFE85QCr1fHk2Cp4G7ByQ== 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.5] 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAkhtZlIb9BGhAAVRP39RcDAj8i+DFnaPfa0w== 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.6] 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAnhtZlo88GGRAAiwbA6AKPAYVFiJ/aLg+38A== 2024-02-21T23:24:30.387 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.7] 2024-02-21T23:24:30.388 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAqhtZlM21WFxAAHzccPGQzH+DjdFbUZPKTNg== 2024-02-21T23:24:30.388 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-02-21T23:24:30.388 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAPhtZllfvxBhAAOdlAB3XH6IFTpbmW/aM3jg== 2024-02-21T23:24:30.390 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.y] 2024-02-21T23:24:30.390 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAOhtZlx1CYExAADw1OdhJ+N/XUVRLQ86IeWg== 2024-02-21T23:24:30.390 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-02-21T23:24:30.390 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAOhtZle8Z5GxAAuhj3K1bupxfYneeGexEONA== 2024-02-21T23:24:30.390 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAOhtZl+CHfIhAA4MaQg9cDkpRovwp4b6CfTg== 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.e] 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAOhtZlERJDKhAARCUqAHRiQSQttiolEwTOjw== 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.0] 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAShtZl3EFcFRAATIHwVh8b6XqW707iZaLHVw== 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.1] 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAVhtZl9Sb1ExAAUeLbOPJ8nrm2x98AlOrFNQ== 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.2] 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAYhtZl9Yw+EhAAaZ7hcv9qb2vdTjWjXL05vw== 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.3] 2024-02-21T23:24:30.391 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAbhtZltqDBEBAAgBOPm19IcB93dxwYMyDE7w== 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.x] 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAOhtZl9xQRGBAAza2kuugJdrQYjyK++VrWUw== 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAOhtZlC0tQLxAAZYKqsmGF6SQDlu5daIM3ZA== 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAOhtZlTOS2NhAAtE9wDO8wq2T9FbGP6mtr+w== 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.f] 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAPhtZla5WRAhAAHDe9igOGbPzB3SCCS6qN9g== 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.4] 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAhhtZlN9lNGxAAyEFE85QCr1fHk2Cp4G7ByQ== 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.5] 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAkhtZlIb9BGhAAVRP39RcDAj8i+DFnaPfa0w== 2024-02-21T23:24:30.392 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.6] 2024-02-21T23:24:30.393 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAnhtZlo88GGRAAiwbA6AKPAYVFiJ/aLg+38A== 2024-02-21T23:24:30.393 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.7] 2024-02-21T23:24:30.393 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAqhtZlM21WFxAAHzccPGQzH+DjdFbUZPKTNg== 2024-02-21T23:24:30.393 INFO:teuthology.orchestra.run.smithi142.stdout:[client.0] 2024-02-21T23:24:30.393 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAPhtZllfvxBhAAOdlAB3XH6IFTpbmW/aM3jg== 2024-02-21T23:24:30.393 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-21T23:24:30.433 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-21T23:24:30.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:30.488 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:30.542 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:30.545 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:30.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:30.602 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:30.656 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:30.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:30.710 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:30.713 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:30.764 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:30.766 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:30.818 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:30.820 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:30.873 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-21T23:24:30.876 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-21T23:24:30.928 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:30.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:30.985 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:30.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:31.042 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:31.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-21T23:24:31.098 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:31.100 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:31.152 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:31.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:31.206 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:31.208 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:31.261 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:31.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-21T23:24:31.316 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-21T23:24:31.318 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-21T23:24:31.373 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-02-21T23:24:31.373 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-02-21T23:24:31.400 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-21T23:24:31.498 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-02-21T23:24:31.524 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-02-21T23:24:31.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-21T23:24:31.648 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-02-21T23:24:31.675 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-02-21T23:24:31.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-21T23:24:31.839 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-02-21T23:24:31.866 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:31.869 DEBUG:teuthology.orchestra.run.smithi142:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-02-21T23:24:31.921 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-02-21T23:24:31.921 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-02-21T23:24:31.921 INFO:tasks.ceph.mon.a:Restarting daemon 2024-02-21T23:24:31.922 DEBUG:teuthology.orchestra.run.smithi133:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2024-02-21T23:24:31.924 INFO:tasks.ceph.mon.a:Started 2024-02-21T23:24:31.924 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-02-21T23:24:31.924 INFO:tasks.ceph.mon.b:Restarting daemon 2024-02-21T23:24:31.924 DEBUG:teuthology.orchestra.run.smithi142:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2024-02-21T23:24:31.964 INFO:tasks.ceph.mon.b:Started 2024-02-21T23:24:31.964 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-02-21T23:24:31.964 INFO:tasks.ceph.mon.c:Restarting daemon 2024-02-21T23:24:31.965 DEBUG:teuthology.orchestra.run.smithi142:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2024-02-21T23:24:31.967 INFO:tasks.ceph.mon.c:Started 2024-02-21T23:24:31.967 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-02-21T23:24:31.967 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-02-21T23:24:31.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2024-02-21T23:24:31.969 INFO:tasks.ceph.mgr.y:Started 2024-02-21T23:24:31.970 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-02-21T23:24:31.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2024-02-21T23:24:31.972 INFO:tasks.ceph.mgr.x:Started 2024-02-21T23:24:31.972 INFO:tasks.ceph:Setting crush tunables to default 2024-02-21T23:24:31.972 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd crush tunables default 2024-02-21T23:24:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.085+0000 7f75e5d8c640 1 Processor -- start 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.085+0000 7f75e5d8c640 1 -- start start 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75e5d8c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 0x7f75e00ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75e5d8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 0x7f75e00c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75e5d8c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e00cd300 con 0x7f75e0070800 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75e5d8c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00cd570 con 0x7f75e0154240 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75e5d8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015a130 con 0x7f75e00703c0 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75dffff640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75e0070800 legacy=0x7f75e015d610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.133:6789/0 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75df7fe640 1 -- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 msgr2=0x7f75e00ccdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:3300/0 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75df7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 0x7f75e00ccdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75d6ffd640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 msgr2=0x7f75e00c90b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3301/0 2024-02-21T23:24:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.086+0000 7f75d6ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 0x7f75e00c90b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-21T23:24:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.287+0000 7f75df7fe640 1 -- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 msgr2=0x7f75e00ccdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:3300/0 2024-02-21T23:24:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.287+0000 7f75dffff640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75e0070800 legacy=0x7f75e015d610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.133:6789/0 2024-02-21T23:24:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.287+0000 7f75df7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 0x7f75e00ccdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-21T23:24:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.287+0000 7f75d6ffd640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 msgr2=0x7f75e00c90b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3301/0 2024-02-21T23:24:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.287+0000 7f75d6ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 0x7f75e00c90b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-21T23:24:32.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.688+0000 7f75df7fe640 1 -- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 msgr2=0x7f75e00ccdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:3300/0 2024-02-21T23:24:32.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.688+0000 7f75dffff640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75e0070800 legacy=0x7f75e015d610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.133:6789/0 2024-02-21T23:24:32.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.688+0000 7f75df7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 0x7f75e00ccdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-21T23:24:32.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.688+0000 7f75d6ffd640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 msgr2=0x7f75e00c90b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3301/0 2024-02-21T23:24:32.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:32.688+0000 7f75d6ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 0x7f75e00c90b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-21T23:24:33.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:33.489+0000 7f75df7fe640 1 -- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 msgr2=0x7f75e00ccdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:3300/0 2024-02-21T23:24:33.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:33.489+0000 7f75dffff640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75e0070800 legacy=0x7f75e015d610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.133:6789/0 2024-02-21T23:24:33.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:33.489+0000 7f75df7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 0x7f75e00ccdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-21T23:24:33.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:33.489+0000 7f75d6ffd640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 msgr2=0x7f75e00c90b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3301/0 2024-02-21T23:24:33.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:33.489+0000 7f75d6ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 0x7f75e00c90b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-21T23:24:35.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75ddffb640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 msgr2=0x7f75e00c90b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75ddffb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75e00703c0 0x7f75e00c90b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75ddffb640 1 -- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 msgr2=0x7f75e00ccdc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75ddffb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75e0154240 0x7f75e00ccdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75ddffb640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75e0070800 legacy=0x7f75e015d610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75ddffb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 0x7f75c0009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75ddffb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e015a410 con 0x7f75c00036c0 2024-02-21T23:24:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75ddffb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00cd570 con 0x7f75c0007370 2024-02-21T23:24:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75df7fe640 1 -- >> v1:172.21.15.142:6789/0 conn(0x7f75c00036c0 legacy=0x7f75c0005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.142:6789/0 2024-02-21T23:24:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75d6ffd640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3300/0 2024-02-21T23:24:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.086+0000 7f75d6ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-21T23:24:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.287+0000 7f75df7fe640 1 -- >> v1:172.21.15.142:6789/0 conn(0x7f75c00036c0 legacy=0x7f75c0005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.142:6789/0 2024-02-21T23:24:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.287+0000 7f75d6ffd640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3300/0 2024-02-21T23:24:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.287+0000 7f75d6ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-21T23:24:35.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.688+0000 7f75df7fe640 1 -- >> v1:172.21.15.142:6789/0 conn(0x7f75c00036c0 legacy=0x7f75c0005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.142:6789/0 2024-02-21T23:24:35.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.688+0000 7f75d6ffd640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3300/0 2024-02-21T23:24:35.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:35.688+0000 7f75d6ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-21T23:24:36.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:36.489+0000 7f75df7fe640 1 -- >> v1:172.21.15.142:6789/0 conn(0x7f75c00036c0 legacy=0x7f75c0005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.142:6789/0 2024-02-21T23:24:36.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:36.489+0000 7f75d6ffd640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3300/0 2024-02-21T23:24:36.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:36.489+0000 7f75d6ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-21T23:24:38.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.086+0000 7f75ddffb640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:38.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.086+0000 7f75ddffb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0007370 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:38.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.086+0000 7f75ddffb640 1 -- >> v1:172.21.15.142:6789/0 conn(0x7f75c00036c0 legacy=0x7f75c0005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:38.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75dffff640 1 -- reap_dead start 2024-02-21T23:24:38.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75ddffb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 0x7f75c0009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:38.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75ddffb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:38.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75ddffb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e00cd300 con 0x7f75c000a350 2024-02-21T23:24:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75ddffb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00cd570 con 0x7f75c0005b70 2024-02-21T23:24:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75ddffb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015a130 con 0x7f75c0009ed0 2024-02-21T23:24:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75d6ffd640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75c000a350 legacy=0x7f75c000a790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.133:6789/0 2024-02-21T23:24:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75df7fe640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3301/0 2024-02-21T23:24:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75dffff640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3300/0 2024-02-21T23:24:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-21T23:24:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.087+0000 7f75dffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-21T23:24:38.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.288+0000 7f75d6ffd640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75c000a350 legacy=0x7f75c000a790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.133:6789/0 2024-02-21T23:24:38.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.288+0000 7f75dffff640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3300/0 2024-02-21T23:24:38.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.288+0000 7f75df7fe640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3301/0 2024-02-21T23:24:38.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.288+0000 7f75dffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-21T23:24:38.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.288+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-21T23:24:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.689+0000 7f75d6ffd640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75c000a350 legacy=0x7f75c000a790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.133:6789/0 2024-02-21T23:24:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.689+0000 7f75dffff640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3300/0 2024-02-21T23:24:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.689+0000 7f75df7fe640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3301/0 2024-02-21T23:24:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.689+0000 7f75dffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-21T23:24:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:38.689+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-21T23:24:39.000 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:00:06.906 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:24:39.218 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:00:07.129 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:24:39.219 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:00:07.131 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:24:39.307 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:00:07.214 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:24:39.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:39.490+0000 7f75d6ffd640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75c000a350 legacy=0x7f75c000a790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.133:6789/0 2024-02-21T23:24:39.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:39.490+0000 7f75dffff640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3300/0 2024-02-21T23:24:39.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:39.490+0000 7f75dffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-21T23:24:39.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:39.490+0000 7f75df7fe640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.142:3301/0 2024-02-21T23:24:39.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:39.490+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-21T23:24:39.538 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:00:07.449 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:24:39.541 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:00:07.452 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:24:41.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.087+0000 7f75ddffb640 1 -- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 msgr2=0x7f75c0009830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:41.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.087+0000 7f75ddffb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0009ed0 0x7f75c0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:41.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.087+0000 7f75ddffb640 1 -- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:41.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.087+0000 7f75ddffb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:41.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.087+0000 7f75ddffb640 1 -- >> v1:172.21.15.133:6789/0 conn(0x7f75c000a350 legacy=0x7f75c000a790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:41.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.087+0000 7f75ddffb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:41.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.087+0000 7f75ddffb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e015a410 con 0x7f75c000e170 2024-02-21T23:24:41.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.087+0000 7f75ddffb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00cd570 con 0x7f75c000d690 2024-02-21T23:24:41.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.194+0000 7f75d6ffd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f75c000e170 0x7f75c0012800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:54032/0 (socket says 172.21.15.133:54032) 2024-02-21T23:24:41.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.194+0000 7f75d6ffd640 1 -- 172.21.15.133:0/3282634343 learned_addr learned my addr 172.21.15.133:0/3282634343 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:24:41.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.208+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.226+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 msgr2=0x7f75c000daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-21T23:24:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.226+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 msgr2=0x7f75c000daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.226+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.226+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-21T23:24:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.568+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.584+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 msgr2=0x7f75c000daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-21T23:24:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.584+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 msgr2=0x7f75c000daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.584+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.584+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-21T23:24:41.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.986+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.987+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 msgr2=0x7f75c000daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-21T23:24:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.987+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 msgr2=0x7f75c000daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.987+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:41.987+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-21T23:24:43.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:43.326+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:43.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:43.327+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 msgr2=0x7f75c000daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-21T23:24:43.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:43.327+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 msgr2=0x7f75c000daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:43.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:43.327+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:43.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:43.327+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-21T23:24:44.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 msgr2=0x7f75c000daa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.088+0000 7f75ddffb640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000d690 0x7f75c000daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 >> v1:172.21.15.142:6789/0 conn(0x7f75c000e170 legacy=0x7f75c0012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.088+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 reap_dead start 2024-02-21T23:24:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.088+0000 7f75ddffb640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e00cd300 con 0x7f75c000a350 2024-02-21T23:24:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c000db60 con 0x7f75c0014040 2024-02-21T23:24:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00cd570 con 0x7f75c0005b70 2024-02-21T23:24:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.088+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:44.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.089+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-02-21T23:24:44.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.089+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:44.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.089+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:44.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.089+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-21T23:24:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.290+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.292+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-02-21T23:24:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.292+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.292+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.292+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-21T23:24:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.693+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.695+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-02-21T23:24:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.695+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.695+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:44.695+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-21T23:24:45.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:45.496+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:45.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:45.498+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-02-21T23:24:45.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:45.498+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:45.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:45.498+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:45.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:45.498+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-21T23:24:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.087+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.087+0000 7f75ddffb640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c0005b70 0x7f75c000d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.087+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 >> v1:172.21.15.142:6789/0 conn(0x7f75c0014040 legacy=0x7f75c0016430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.087+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 >> v1:172.21.15.133:6789/0 conn(0x7f75c000a350 legacy=0x7f75c000a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.087+0000 7f75ddffb640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.087+0000 7f75ddffb640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.087+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00cd570 con 0x7f75c00078a0 2024-02-21T23:24:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.087+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e015a130 con 0x7f75c00082c0 2024-02-21T23:24:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.090+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:47.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.090+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:47.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.091+0000 7f75d6ffd640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 msgr2=0x7f75c0012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-21T23:24:47.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.091+0000 7f75d6ffd640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 msgr2=0x7f75c0012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:47.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.091+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:47.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.091+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-21T23:24:47.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.091+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 msgr2=0x7f75c0007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-02-21T23:24:47.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.091+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 msgr2=0x7f75c0007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:47.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.091+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:47.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.091+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-21T23:24:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.292+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.293+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.293+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 msgr2=0x7f75c0007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-02-21T23:24:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.293+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 msgr2=0x7f75c0007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.293+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.293+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-21T23:24:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.294+0000 7f75d6ffd640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 msgr2=0x7f75c0012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-21T23:24:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.294+0000 7f75d6ffd640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 msgr2=0x7f75c0012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.294+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.294+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-21T23:24:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.694+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.695+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:47.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.696+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 msgr2=0x7f75c0007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-21T23:24:47.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.696+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 msgr2=0x7f75c0007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:47.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.696+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:47.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.696+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-21T23:24:47.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.697+0000 7f75d6ffd640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 msgr2=0x7f75c0012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-02-21T23:24:47.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.697+0000 7f75d6ffd640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 msgr2=0x7f75c0012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:47.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.697+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:47.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:47.697+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-21T23:24:48.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.499+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:48.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.500+0000 7f75d6ffd640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 msgr2=0x7f75c0012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-02-21T23:24:48.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.500+0000 7f75d6ffd640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 msgr2=0x7f75c0012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:48.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.500+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:48.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.500+0000 7f75d6ffd640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-21T23:24:48.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.529+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.537+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 msgr2=0x7f75c0007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-21T23:24:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.537+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 msgr2=0x7f75c0007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-21T23:24:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.537+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-21T23:24:48.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:48.537+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-21T23:24:50.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 msgr2=0x7f75c0012800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:50.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75ddffb640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c00082c0 0x7f75c0012800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 msgr2=0x7f75c0007c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-21T23:24:50.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75ddffb640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c00078a0 0x7f75c0007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75dffff640 1 -- 172.21.15.133:0/3282634343 reap_dead start 2024-02-21T23:24:50.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75ddffb640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 0x7f75c000a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:50.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75ddffb640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000a2b0 0x7f75c000d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:50.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c000de10 con 0x7f75c00082c0 2024-02-21T23:24:50.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00cd570 con 0x7f75c000a2b0 2024-02-21T23:24:50.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75ddffb640 1 -- 172.21.15.133:0/3282634343 --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e015a130 con 0x7f75c0005b70 2024-02-21T23:24:50.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.088+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.142:3300/0 conn(0x7f75c0005b70 0x7f75c000a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:50.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.089+0000 7f75dffff640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000a2b0 0x7f75c000d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:50.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.091+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000a2b0 msgr2=0x7f75c000d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:50.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.091+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000a2b0 0x7f75c000d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.091+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 >> v1:172.21.15.133:6789/0 conn(0x7f75c00082c0 legacy=0x7f75c00078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:50.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.091+0000 7f75df7fe640 1 -- 172.21.15.133:0/3282634343 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c0007fb0 con 0x7f75c0005b70 2024-02-21T23:24:50.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.092+0000 7f75df7fe640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c0005b70 0x7f75c000a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75e00ccd50 tx=0x7f75d002e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d641b95590f322b server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:50.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.095+0000 7f75deffd640 1 -- 172.21.15.133:0/3282634343 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0034070 con 0x7f75c0005b70 2024-02-21T23:24:50.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.098+0000 7f75deffd640 1 -- 172.21.15.133:0/3282634343 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0034420 con 0x7f75c0005b70 2024-02-21T23:24:50.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.098+0000 7f75deffd640 1 -- 172.21.15.133:0/3282634343 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0034810 con 0x7f75c0005b70 2024-02-21T23:24:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.099+0000 7f75e5d8c640 1 -- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c0005b70 msgr2=0x7f75c000a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.099+0000 7f75e5d8c640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c0005b70 0x7f75c000a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75e00ccd50 tx=0x7f75d002e720 comp rx=0 tx=0).stop 2024-02-21T23:24:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.099+0000 7f75e5d8c640 1 -- 172.21.15.133:0/3282634343 shutdown_connections 2024-02-21T23:24:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.099+0000 7f75e5d8c640 1 --2- 172.21.15.133:0/3282634343 >> v2:172.21.15.133:3300/0 conn(0x7f75c000a2b0 0x7f75c000d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.099+0000 7f75e5d8c640 1 --2- 172.21.15.133:0/3282634343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c0005b70 0x7f75c000a790 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.099+0000 7f75e5d8c640 1 -- 172.21.15.133:0/3282634343 >> 172.21.15.133:0/3282634343 conn(0x7f75e0061b50 msgr2=0x7f75e0072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.099+0000 7f75e5d8c640 1 -- 172.21.15.133:0/3282634343 shutdown_connections 2024-02-21T23:24:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.099+0000 7f75e5d8c640 1 -- 172.21.15.133:0/3282634343 wait complete. 2024-02-21T23:24:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.100+0000 7f75e5d8c640 1 Processor -- start 2024-02-21T23:24:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.100+0000 7f75e5d8c640 1 -- start start 2024-02-21T23:24:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.101+0000 7f75e5d8c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0005b70 0x7f75e00cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.101+0000 7f75e5d8c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c00082c0 0x7f75e0145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.101+0000 7f75e5d8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c000a2b0 0x7f75e0158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.101+0000 7f75e5d8c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0154620 con 0x7f75c0005b70 2024-02-21T23:24:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.101+0000 7f75e5d8c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015a9d0 con 0x7f75c00082c0 2024-02-21T23:24:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.101+0000 7f75e5d8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015c730 con 0x7f75c000a2b0 2024-02-21T23:24:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.102+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0005b70 0x7f75e00cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:50.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.102+0000 7f75d6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c00082c0 0x7f75e0145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:50.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.102+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0005b70 0x7f75e00cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59534/0 (socket says 172.21.15.133:59534) 2024-02-21T23:24:50.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.102+0000 7f75df7fe640 1 -- 172.21.15.133:0/2064191959 learned_addr learned my addr 172.21.15.133:0/2064191959 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:24:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.105+0000 7f75df7fe640 1 -- 172.21.15.133:0/2064191959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c000a2b0 msgr2=0x7f75e0158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.105+0000 7f75df7fe640 1 --2- 172.21.15.133:0/2064191959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c000a2b0 0x7f75e0158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.105+0000 7f75df7fe640 1 -- 172.21.15.133:0/2064191959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c00082c0 msgr2=0x7f75e0145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.105+0000 7f75df7fe640 1 --2- 172.21.15.133:0/2064191959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c00082c0 0x7f75e0145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.105+0000 7f75df7fe640 1 -- 172.21.15.133:0/2064191959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00cb3a0 con 0x7f75c0005b70 2024-02-21T23:24:50.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.107+0000 7f75df7fe640 1 --2- 172.21.15.133:0/2064191959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0005b70 0x7f75e00cbb30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f75d002e760 tx=0x7f75d0005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:50.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.111+0000 7f75dcff9640 1 -- 172.21.15.133:0/2064191959 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0047070 con 0x7f75c0005b70 2024-02-21T23:24:50.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.111+0000 7f75dcff9640 1 -- 172.21.15.133:0/2064191959 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0043650 con 0x7f75c0005b70 2024-02-21T23:24:50.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.111+0000 7f75dcff9640 1 -- 172.21.15.133:0/2064191959 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0034ed0 con 0x7f75c0005b70 2024-02-21T23:24:50.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.111+0000 7f75e5d8c640 1 -- 172.21.15.133:0/2064191959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e014e000 con 0x7f75c0005b70 2024-02-21T23:24:50.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.111+0000 7f75e5d8c640 1 -- 172.21.15.133:0/2064191959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e00cb610 con 0x7f75c0005b70 2024-02-21T23:24:50.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.112+0000 7f75e5d8c640 1 -- 172.21.15.133:0/2064191959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e00c7280 con 0x7f75c0005b70 2024-02-21T23:24:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.115+0000 7f75dcff9640 1 -- 172.21.15.133:0/2064191959 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f75d004a070 con 0x7f75c0005b70 2024-02-21T23:24:50.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.115+0000 7f75dcff9640 1 -- 172.21.15.133:0/2064191959 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f75d00566f0 con 0x7f75c0005b70 2024-02-21T23:24:50.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.275+0000 7f75dcff9640 1 -- 172.21.15.133:0/2064191959 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f75e00c7280 con 0x7f75c0005b70 2024-02-21T23:24:50.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.339+0000 7f75e5d8c640 1 -- 172.21.15.133:0/2064191959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f75e006a130 con 0x7f75c0005b70 2024-02-21T23:24:50.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.469+0000 7f75dcff9640 1 -- 172.21.15.133:0/2064191959 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f75d0043a00 con 0x7f75c0005b70 2024-02-21T23:24:50.456 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2024-02-21T23:24:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.470+0000 7f75e5d8c640 1 -- 172.21.15.133:0/2064191959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0005b70 msgr2=0x7f75e00cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.470+0000 7f75e5d8c640 1 --2- 172.21.15.133:0/2064191959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0005b70 0x7f75e00cbb30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f75d002e760 tx=0x7f75d0005c30 comp rx=0 tx=0).stop 2024-02-21T23:24:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.470+0000 7f75e5d8c640 1 -- 172.21.15.133:0/2064191959 shutdown_connections 2024-02-21T23:24:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.470+0000 7f75e5d8c640 1 --2- 172.21.15.133:0/2064191959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75c000a2b0 0x7f75e0158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.470+0000 7f75e5d8c640 1 --2- 172.21.15.133:0/2064191959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75c00082c0 0x7f75e0145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.470+0000 7f75e5d8c640 1 --2- 172.21.15.133:0/2064191959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0005b70 0x7f75e00cbb30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.470+0000 7f75e5d8c640 1 -- 172.21.15.133:0/2064191959 >> 172.21.15.133:0/2064191959 conn(0x7f75e0061b50 msgr2=0x7f75e0149570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.470+0000 7f75e5d8c640 1 -- 172.21.15.133:0/2064191959 shutdown_connections 2024-02-21T23:24:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:50.470+0000 7f75e5d8c640 1 -- 172.21.15.133:0/2064191959 wait complete. 2024-02-21T23:24:50.469 INFO:tasks.ceph:check_enable_crimson: False 2024-02-21T23:24:50.469 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-02-21T23:24:50.469 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:50.469 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-02-21T23:24:50.472 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:50.485+0000 7f0e50021040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-21T23:24:50.496 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:50.496 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-02-21T23:24:50.530 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:50.544+0000 7f0e50021040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-21T23:24:50.564 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:50.564 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-02-21T23:24:50.633 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:24:50.633 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-02-21T23:24:50.675 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:50.688+0000 7f8a1d6da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-21T23:24:50.701 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:50.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-02-21T23:24:50.729 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:50.729 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-02-21T23:24:50.731 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:50.744+0000 7f8a1d6da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-21T23:24:50.795 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:50.796 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-02-21T23:24:50.862 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-21T23:24:50.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-02-21T23:24:50.929 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new dd65829d-e81e-4ce3-82a8-0232e6ec2d06 0 2024-02-21T23:24:51.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.085+0000 7fcbbc836640 1 Processor -- start 2024-02-21T23:24:51.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.085+0000 7fcbbc836640 1 -- start start 2024-02-21T23:24:51.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.085+0000 7fcbbc836640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 0x7fcbb80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.085+0000 7fcbbc836640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcbb80c41a0 0x7fcbb8070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.085+0000 7fcbbc836640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb8156e90 con 0x7fcbb8150d30 2024-02-21T23:24:51.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.086+0000 7fcbbc836640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb8157150 con 0x7fcbb80c41a0 2024-02-21T23:24:51.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.086+0000 7fcbbc836640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8157430 con 0x7fcbb80c3dc0 2024-02-21T23:24:51.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.086+0000 7fcbb659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 0x7fcbb80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:51.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.086+0000 7fcbb5d9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcbb80c41a0 0x7fcbb8070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:51.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.086+0000 7fcbb6d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcbb8150d30 0x7fcbb815a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.142:57718/0 (socket says 172.21.15.142:57718) 2024-02-21T23:24:51.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.086+0000 7fcbb6d9d640 1 -- 172.21.15.142:0/3398724469 learned_addr learned my addr 172.21.15.142:0/3398724469 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-02-21T23:24:51.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.089+0000 7fcbb659c640 1 -- 172.21.15.142:0/3398724469 >> v2:172.21.15.142:3300/0 conn(0x7fcbb80c41a0 msgr2=0x7fcbb8070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:51.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.089+0000 7fcbb659c640 1 --2- 172.21.15.142:0/3398724469 >> v2:172.21.15.142:3300/0 conn(0x7fcbb80c41a0 0x7fcbb8070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.089+0000 7fcbb659c640 1 -- 172.21.15.142:0/3398724469 >> v1:172.21.15.133:6789/0 conn(0x7fcbb8150d30 legacy=0x7fcbb815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:51.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.089+0000 7fcbb659c640 1 -- 172.21.15.142:0/3398724469 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb8157710 con 0x7fcbb80c3dc0 2024-02-21T23:24:51.077 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.091+0000 7fcbb659c640 1 --2- 172.21.15.142:0/3398724469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 0x7fcbb80cd010 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcba80097a0 tx=0x7fcba802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f57bba8f4641ba7 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:51.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.097+0000 7fcbb559a640 1 -- 172.21.15.142:0/3398724469 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8034070 con 0x7fcbb80c3dc0 2024-02-21T23:24:51.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.097+0000 7fcbb559a640 1 -- 172.21.15.142:0/3398724469 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba8030610 con 0x7fcbb80c3dc0 2024-02-21T23:24:51.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.097+0000 7fcbb559a640 1 -- 172.21.15.142:0/3398724469 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80309e0 con 0x7fcbb80c3dc0 2024-02-21T23:24:51.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.098+0000 7fcbbc836640 1 -- 172.21.15.142:0/3398724469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 msgr2=0x7fcbb80cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:51.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.098+0000 7fcbbc836640 1 --2- 172.21.15.142:0/3398724469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 0x7fcbb80cd010 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcba80097a0 tx=0x7fcba802b960 comp rx=0 tx=0).stop 2024-02-21T23:24:51.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.098+0000 7fcbbc836640 1 -- 172.21.15.142:0/3398724469 shutdown_connections 2024-02-21T23:24:51.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.098+0000 7fcbbc836640 1 --2- 172.21.15.142:0/3398724469 >> v2:172.21.15.142:3300/0 conn(0x7fcbb80c41a0 0x7fcbb8070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.098+0000 7fcbbc836640 1 --2- 172.21.15.142:0/3398724469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 0x7fcbb80cd010 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.098+0000 7fcbbc836640 1 -- 172.21.15.142:0/3398724469 >> 172.21.15.142:0/3398724469 conn(0x7fcbb8061b50 msgr2=0x7fcbb8072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:51.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.098+0000 7fcbbc836640 1 -- 172.21.15.142:0/3398724469 shutdown_connections 2024-02-21T23:24:51.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.098+0000 7fcbbc836640 1 -- 172.21.15.142:0/3398724469 wait complete. 2024-02-21T23:24:51.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.100+0000 7fcbbc836640 1 Processor -- start 2024-02-21T23:24:51.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.100+0000 7fcbbc836640 1 -- start start 2024-02-21T23:24:51.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.100+0000 7fcbbc836640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 0x7fcbb8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.100+0000 7fcbbc836640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb80c41a0 0x7fcbb8154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.101+0000 7fcbbc836640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb8150d30 0x7fcbb8154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.101+0000 7fcbbc836640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8070190 con 0x7fcbb8150d30 2024-02-21T23:24:51.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.101+0000 7fcbbc836640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8157cd0 con 0x7fcbb80c41a0 2024-02-21T23:24:51.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.101+0000 7fcbbc836640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8070950 con 0x7fcbb80c3dc0 2024-02-21T23:24:51.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.101+0000 7fcbb5d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb80c41a0 0x7fcbb8154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:51.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.101+0000 7fcbb6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb8150d30 0x7fcbb8154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:51.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.101+0000 7fcbb6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb8150d30 0x7fcbb8154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.142:58782/0 (socket says 172.21.15.142:58782) 2024-02-21T23:24:51.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.101+0000 7fcbb6d9d640 1 -- 172.21.15.142:0/920125740 learned_addr learned my addr 172.21.15.142:0/920125740 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:51.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.105+0000 7fcbb6d9d640 1 -- 172.21.15.142:0/920125740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 msgr2=0x7fcbb8152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:51.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.105+0000 7fcbb6d9d640 1 --2- 172.21.15.142:0/920125740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 0x7fcbb8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.105+0000 7fcbb6d9d640 1 -- 172.21.15.142:0/920125740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb80c41a0 msgr2=0x7fcbb8154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:51.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.105+0000 7fcbb6d9d640 1 --2- 172.21.15.142:0/920125740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb80c41a0 0x7fcbb8154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.105+0000 7fcbb6d9d640 1 -- 172.21.15.142:0/920125740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb81ed7a0 con 0x7fcbb8150d30 2024-02-21T23:24:51.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.107+0000 7fcbb6d9d640 1 --2- 172.21.15.142:0/920125740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb8150d30 0x7fcbb8154750 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00021c0 tx=0x7fcbb00021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:51.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.110+0000 7fcba6ffd640 1 -- 172.21.15.142:0/920125740 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0015070 con 0x7fcbb8150d30 2024-02-21T23:24:51.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.110+0000 7fcba6ffd640 1 -- 172.21.15.142:0/920125740 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb000acd0 con 0x7fcbb8150d30 2024-02-21T23:24:51.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.110+0000 7fcbbc836640 1 -- 172.21.15.142:0/920125740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb81eda10 con 0x7fcbb8150d30 2024-02-21T23:24:51.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.110+0000 7fcba6ffd640 1 -- 172.21.15.142:0/920125740 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0011670 con 0x7fcbb8150d30 2024-02-21T23:24:51.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.110+0000 7fcbbc836640 1 -- 172.21.15.142:0/920125740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb8153cf0 con 0x7fcbb8150d30 2024-02-21T23:24:51.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.111+0000 7fcbbc836640 1 -- 172.21.15.142:0/920125740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb806a130 con 0x7fcbb8150d30 2024-02-21T23:24:51.126 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.140+0000 7f0e50021040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-21T23:24:51.180 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.194+0000 7f0e50021040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-21T23:24:51.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.209+0000 7fcba6ffd640 1 -- 172.21.15.142:0/920125740 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcbb0018070 con 0x7fcbb8150d30 2024-02-21T23:24:51.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.213+0000 7fcba6ffd640 1 -- 172.21.15.142:0/920125740 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcbb0024b30 con 0x7fcbb8150d30 2024-02-21T23:24:51.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.213+0000 7fcba6ffd640 1 -- 172.21.15.142:0/920125740 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fcbb0024d70 con 0x7fcbb8150d30 2024-02-21T23:24:51.227 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.241+0000 7f0e50021040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-21T23:24:51.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.287+0000 7fcbbc836640 1 -- 172.21.15.142:0/920125740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dd65829d-e81e-4ce3-82a8-0232e6ec2d06", "id": 0} v 0) v1 -- 0x7fcbb8153ec0 con 0x7fcbb8150d30 2024-02-21T23:24:51.278 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.291+0000 7f8a1d6da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-21T23:24:51.283 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.297+0000 7f0e50021040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-21T23:24:51.325 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.339+0000 7f0e50021040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-21T23:24:51.336 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.349+0000 7f8a1d6da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-21T23:24:51.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.374+0000 7fcba6ffd640 1 -- 172.21.15.142:0/920125740 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd65829d-e81e-4ce3-82a8-0232e6ec2d06", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcbb0011810 con 0x7fcbb8150d30 2024-02-21T23:24:51.361 INFO:teuthology.orchestra.run.smithi142.stdout:0 2024-02-21T23:24:51.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.375+0000 7fcbbc836640 1 -- 172.21.15.142:0/920125740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb8150d30 msgr2=0x7fcbb8154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:51.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.375+0000 7fcbbc836640 1 --2- 172.21.15.142:0/920125740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb8150d30 0x7fcbb8154750 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00021c0 tx=0x7fcbb00021f0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.375+0000 7fcbbc836640 1 -- 172.21.15.142:0/920125740 shutdown_connections 2024-02-21T23:24:51.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.375+0000 7fcbbc836640 1 --2- 172.21.15.142:0/920125740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb8150d30 0x7fcbb8154750 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.375+0000 7fcbbc836640 1 --2- 172.21.15.142:0/920125740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb80c41a0 0x7fcbb8154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.375+0000 7fcbbc836640 1 --2- 172.21.15.142:0/920125740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbb80c3dc0 0x7fcbb8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.375+0000 7fcbbc836640 1 -- 172.21.15.142:0/920125740 >> 172.21.15.142:0/920125740 conn(0x7fcbb8061b50 msgr2=0x7fcbb80c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:51.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.376+0000 7fcbbc836640 1 -- 172.21.15.142:0/920125740 shutdown_connections 2024-02-21T23:24:51.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.376+0000 7fcbbc836640 1 -- 172.21.15.142:0/920125740 wait complete. 2024-02-21T23:24:51.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 8b21a887-709e-4de1-8f09-0e0dffd47b78 1 2024-02-21T23:24:51.399 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.412+0000 7f8a1d6da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-21T23:24:51.451 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.464+0000 7f0e50021040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-21T23:24:51.456 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.469+0000 7f8a1d6da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-21T23:24:51.478 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.491+0000 7fe107e36640 1 Processor -- start 2024-02-21T23:24:51.478 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.491+0000 7fe107e36640 1 -- start start 2024-02-21T23:24:51.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.492+0000 7fe107e36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 0x7fe1000ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.492+0000 7fe107e36640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe100070800 0x7fe10015d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.492+0000 7fe107e36640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1000cd300 con 0x7fe1000703c0 2024-02-21T23:24:51.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.492+0000 7fe107e36640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1000cd570 con 0x7fe100070800 2024-02-21T23:24:51.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.492+0000 7fe107e36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10015a130 con 0x7fe100154240 2024-02-21T23:24:51.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.493+0000 7fe1063d3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe100070800 0x7fe10015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:51.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.493+0000 7fe105bd2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 0x7fe1000ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:51.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.493+0000 7fe1063d3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe100070800 0x7fe10015d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42104/0 (socket says 172.21.15.142:42104) 2024-02-21T23:24:51.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.493+0000 7fe1063d3640 1 -- 172.21.15.142:0/1224403644 learned_addr learned my addr 172.21.15.142:0/1224403644 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:51.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.495+0000 7fe1063d3640 1 -- 172.21.15.142:0/1224403644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 msgr2=0x7fe1000ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:51.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.495+0000 7fe1063d3640 1 --2- 172.21.15.142:0/1224403644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 0x7fe1000ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.495+0000 7fe1063d3640 1 -- 172.21.15.142:0/1224403644 >> v1:172.21.15.142:6789/0 conn(0x7fe1000703c0 legacy=0x7fe1000c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:51.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.495+0000 7fe1063d3640 1 -- 172.21.15.142:0/1224403644 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10015a410 con 0x7fe100070800 2024-02-21T23:24:51.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.497+0000 7fe1063d3640 1 --2- 172.21.15.142:0/1224403644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 0x7fe10015d610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009e50 tx=0x7fe0f802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a580e65c4d26618c server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:51.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.503+0000 7fe104bd0640 1 -- 172.21.15.142:0/1224403644 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f802ca70 con 0x7fe100070800 2024-02-21T23:24:51.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.503+0000 7fe104bd0640 1 -- 172.21.15.142:0/1224403644 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f802cc10 con 0x7fe100070800 2024-02-21T23:24:51.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.503+0000 7fe104bd0640 1 -- 172.21.15.142:0/1224403644 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80343f0 con 0x7fe100070800 2024-02-21T23:24:51.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.504+0000 7fe107e36640 1 -- 172.21.15.142:0/1224403644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 msgr2=0x7fe10015d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:51.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.504+0000 7fe107e36640 1 --2- 172.21.15.142:0/1224403644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 0x7fe10015d610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009e50 tx=0x7fe0f802bed0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.504+0000 7fe107e36640 1 -- 172.21.15.142:0/1224403644 shutdown_connections 2024-02-21T23:24:51.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.504+0000 7fe107e36640 1 --2- 172.21.15.142:0/1224403644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 0x7fe10015d610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.504+0000 7fe107e36640 1 --2- 172.21.15.142:0/1224403644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 0x7fe1000ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.504+0000 7fe107e36640 1 -- 172.21.15.142:0/1224403644 >> 172.21.15.142:0/1224403644 conn(0x7fe100061b50 msgr2=0x7fe100072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:51.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.504+0000 7fe107e36640 1 -- 172.21.15.142:0/1224403644 shutdown_connections 2024-02-21T23:24:51.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.504+0000 7fe107e36640 1 -- 172.21.15.142:0/1224403644 wait complete. 2024-02-21T23:24:51.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.505+0000 7fe107e36640 1 Processor -- start 2024-02-21T23:24:51.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.505+0000 7fe107e36640 1 -- start start 2024-02-21T23:24:51.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.507+0000 7fe107e36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1000703c0 0x7fe1000c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.507+0000 7fe107e36640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 0x7fe1000ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.507+0000 7fe107e36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 0x7fe1001efb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:51.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.507+0000 7fe107e36640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100154620 con 0x7fe1000703c0 2024-02-21T23:24:51.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.507+0000 7fe107e36640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10015a9d0 con 0x7fe100070800 2024-02-21T23:24:51.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.507+0000 7fe107e36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001f2e00 con 0x7fe100154240 2024-02-21T23:24:51.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.507+0000 7fe1063d3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 0x7fe1001efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:51.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.507+0000 7fe1053d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 0x7fe1000ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:51.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.508+0000 7fe1063d3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 0x7fe1001efb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:39914/0 (socket says 172.21.15.142:39914) 2024-02-21T23:24:51.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.508+0000 7fe1053d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 0x7fe1000ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42112/0 (socket says 172.21.15.142:42112) 2024-02-21T23:24:51.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.508+0000 7fe1063d3640 1 -- 172.21.15.142:0/3408728566 learned_addr learned my addr 172.21.15.142:0/3408728566 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:51.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.508+0000 7fe105bd2640 1 --2- 172.21.15.142:0/3408728566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1000703c0 0x7fe1000c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:51.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.511+0000 7fe1053d1640 1 -- 172.21.15.142:0/3408728566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 msgr2=0x7fe1001efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:51.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.511+0000 7fe1053d1640 1 --2- 172.21.15.142:0/3408728566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 0x7fe1001efb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.511+0000 7fe1053d1640 1 -- 172.21.15.142:0/3408728566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1000703c0 msgr2=0x7fe1000c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:51.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.511+0000 7fe1053d1640 1 --2- 172.21.15.142:0/3408728566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1000703c0 0x7fe1000c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.511+0000 7fe1053d1640 1 -- 172.21.15.142:0/3408728566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1001f30e0 con 0x7fe100070800 2024-02-21T23:24:51.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.514+0000 7fe1053d1640 1 --2- 172.21.15.142:0/3408728566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 0x7fe1000ca570 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b3e0 tx=0x7fe0fc00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:51.501 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.515+0000 7f8a1d6da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-21T23:24:51.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.520+0000 7fe0f67fc640 1 -- 172.21.15.142:0/3408728566 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc015020 con 0x7fe100070800 2024-02-21T23:24:51.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.520+0000 7fe0f67fc640 1 -- 172.21.15.142:0/3408728566 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc011830 con 0x7fe100070800 2024-02-21T23:24:51.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.520+0000 7fe0f67fc640 1 -- 172.21.15.142:0/3408728566 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc011c20 con 0x7fe100070800 2024-02-21T23:24:51.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.520+0000 7fe107e36640 1 -- 172.21.15.142:0/3408728566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1001f33c0 con 0x7fe100070800 2024-02-21T23:24:51.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.520+0000 7fe107e36640 1 -- 172.21.15.142:0/3408728566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1001f3980 con 0x7fe100070800 2024-02-21T23:24:51.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.521+0000 7fe107e36640 1 -- 172.21.15.142:0/3408728566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10006a130 con 0x7fe100070800 2024-02-21T23:24:51.530 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.544+0000 7f0e50021040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-21T23:24:51.578 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.592+0000 7f0e50021040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-21T23:24:51.622 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.635+0000 7f8a1d6da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-21T23:24:51.632 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.646+0000 7f0e50021040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-21T23:24:51.681 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:51.695+0000 7f0e50021040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-21T23:24:51.696 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.709+0000 7f8a1d6da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-21T23:24:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.718+0000 7fe0f67fc640 1 -- 172.21.15.142:0/3408728566 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe0fc018020 con 0x7fe100070800 2024-02-21T23:24:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.718+0000 7fe0f67fc640 1 -- 172.21.15.142:0/3408728566 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fe0fc01a830 con 0x7fe100070800 2024-02-21T23:24:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.718+0000 7fe0f67fc640 1 -- 172.21.15.142:0/3408728566 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe0fc01f020 con 0x7fe100070800 2024-02-21T23:24:51.744 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.758+0000 7f8a1d6da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-21T23:24:51.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.794+0000 7fe107e36640 1 -- 172.21.15.142:0/3408728566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8b21a887-709e-4de1-8f09-0e0dffd47b78", "id": 1} v 0) v1 -- 0x7fe1000c9980 con 0x7fe100070800 2024-02-21T23:24:51.799 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.812+0000 7f8a1d6da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-21T23:24:51.848 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:51.861+0000 7f8a1d6da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-21T23:24:51.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.910+0000 7fe0f67fc640 1 -- 172.21.15.142:0/3408728566 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8b21a887-709e-4de1-8f09-0e0dffd47b78", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe0fc011dc0 con 0x7fe100070800 2024-02-21T23:24:51.897 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-02-21T23:24:51.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.911+0000 7fe107e36640 1 -- 172.21.15.142:0/3408728566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 msgr2=0x7fe1000ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:51.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.911+0000 7fe107e36640 1 --2- 172.21.15.142:0/3408728566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 0x7fe1000ca570 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b3e0 tx=0x7fe0fc00b8a0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.911+0000 7fe107e36640 1 -- 172.21.15.142:0/3408728566 shutdown_connections 2024-02-21T23:24:51.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.911+0000 7fe107e36640 1 --2- 172.21.15.142:0/3408728566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe100154240 0x7fe1001efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.911+0000 7fe107e36640 1 --2- 172.21.15.142:0/3408728566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe100070800 0x7fe1000ca570 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.911+0000 7fe107e36640 1 --2- 172.21.15.142:0/3408728566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1000703c0 0x7fe1000c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:51.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.911+0000 7fe107e36640 1 -- 172.21.15.142:0/3408728566 >> 172.21.15.142:0/3408728566 conn(0x7fe100061b50 msgr2=0x7fe1000c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:51.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.911+0000 7fe107e36640 1 -- 172.21.15.142:0/3408728566 shutdown_connections 2024-02-21T23:24:51.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:51.911+0000 7fe107e36640 1 -- 172.21.15.142:0/3408728566 wait complete. 2024-02-21T23:24:51.909 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 9afcbb30-e40c-4c41-83c2-6886fcb8ee78 2 2024-02-21T23:24:52.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.026+0000 7f11b271e640 1 Processor -- start 2024-02-21T23:24:52.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.026+0000 7f11b271e640 1 -- start start 2024-02-21T23:24:52.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.026+0000 7f11b271e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0cdfe0 0x7f11ac0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.026+0000 7f11b271e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11ac0ce3c0 0x7f11ac070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.026+0000 7f11b271e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11ac0712b0 0x7f11ac14e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.027+0000 7f11b271e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ac0c84b0 con 0x7f11ac0ce3c0 2024-02-21T23:24:52.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.027+0000 7f11b271e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ac158ca0 con 0x7f11ac0712b0 2024-02-21T23:24:52.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.027+0000 7f11b271e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0717e0 con 0x7f11ac0cdfe0 2024-02-21T23:24:52.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.027+0000 7f11b0cbb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11ac0712b0 0x7f11ac14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.027+0000 7f11ab7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11ac0ce3c0 0x7f11ac070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.027+0000 7f11abfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0cdfe0 0x7f11ac0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.028+0000 7f11b0cbb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11ac0712b0 0x7f11ac14e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42118/0 (socket says 172.21.15.142:42118) 2024-02-21T23:24:52.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.028+0000 7f11b0cbb640 1 -- 172.21.15.142:0/21291437 learned_addr learned my addr 172.21.15.142:0/21291437 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:52.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.030+0000 7f11b0cbb640 1 -- 172.21.15.142:0/21291437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0cdfe0 msgr2=0x7f11ac0c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:52.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.030+0000 7f11b0cbb640 1 --2- 172.21.15.142:0/21291437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0cdfe0 0x7f11ac0c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.030+0000 7f11b0cbb640 1 -- 172.21.15.142:0/21291437 >> v2:172.21.15.133:3300/0 conn(0x7f11ac0ce3c0 msgr2=0x7f11ac070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:52.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.030+0000 7f11b0cbb640 1 --2- 172.21.15.142:0/21291437 >> v2:172.21.15.133:3300/0 conn(0x7f11ac0ce3c0 0x7f11ac070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.030+0000 7f11b0cbb640 1 -- 172.21.15.142:0/21291437 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac158090 con 0x7f11ac0712b0 2024-02-21T23:24:52.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.032+0000 7f11b0cbb640 1 --2- 172.21.15.142:0/21291437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac0712b0 0x7f11ac14e510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f11ac155b50 tx=0x7f11a402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9080ed0bbe944af server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:52.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.035+0000 7f11aaffd640 1 -- 172.21.15.142:0/21291437 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a402ca70 con 0x7f11ac0712b0 2024-02-21T23:24:52.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.035+0000 7f11aaffd640 1 -- 172.21.15.142:0/21291437 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a402cc10 con 0x7f11ac0712b0 2024-02-21T23:24:52.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.035+0000 7f11aaffd640 1 -- 172.21.15.142:0/21291437 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a40343f0 con 0x7f11ac0712b0 2024-02-21T23:24:52.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.036+0000 7f11b271e640 1 -- 172.21.15.142:0/21291437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac0712b0 msgr2=0x7f11ac14e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.036+0000 7f11b271e640 1 --2- 172.21.15.142:0/21291437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac0712b0 0x7f11ac14e510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f11ac155b50 tx=0x7f11a402bed0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.036+0000 7f11b271e640 1 -- 172.21.15.142:0/21291437 shutdown_connections 2024-02-21T23:24:52.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.036+0000 7f11b271e640 1 --2- 172.21.15.142:0/21291437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac0712b0 0x7f11ac14e510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.036+0000 7f11b271e640 1 --2- 172.21.15.142:0/21291437 >> v2:172.21.15.133:3300/0 conn(0x7f11ac0ce3c0 0x7f11ac070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.036+0000 7f11b271e640 1 --2- 172.21.15.142:0/21291437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0cdfe0 0x7f11ac0c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.036+0000 7f11b271e640 1 -- 172.21.15.142:0/21291437 >> 172.21.15.142:0/21291437 conn(0x7f11ac06aec0 msgr2=0x7f11ac06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:52.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.036+0000 7f11b271e640 1 -- 172.21.15.142:0/21291437 shutdown_connections 2024-02-21T23:24:52.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.036+0000 7f11b271e640 1 -- 172.21.15.142:0/21291437 wait complete. 2024-02-21T23:24:52.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.038+0000 7f11b271e640 1 Processor -- start 2024-02-21T23:24:52.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.038+0000 7f11b271e640 1 -- start start 2024-02-21T23:24:52.025 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.038+0000 7f11b271e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac0712b0 0x7f11ac0cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.038+0000 7f11b271e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac0cdfe0 0x7f11ac0c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.039+0000 7f11b271e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0ce3c0 0x7f11ac0c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.039+0000 7f11b271e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac158930 con 0x7f11ac0712b0 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.039+0000 7f11b271e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac158650 con 0x7f11ac0cdfe0 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.039+0000 7f11b271e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac14ea50 con 0x7f11ac0ce3c0 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.039+0000 7f11b0cbb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0ce3c0 0x7f11ac0c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.039+0000 7f11ab7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac0cdfe0 0x7f11ac0c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.039+0000 7f11abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac0712b0 0x7f11ac0cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.039+0000 7f11abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac0712b0 0x7f11ac0cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.142:58810/0 (socket says 172.21.15.142:58810) 2024-02-21T23:24:52.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.039+0000 7f11abfff640 1 -- 172.21.15.142:0/3303383916 learned_addr learned my addr 172.21.15.142:0/3303383916 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:52.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.043+0000 7f11abfff640 1 -- 172.21.15.142:0/3303383916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0ce3c0 msgr2=0x7f11ac0c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.043+0000 7f11abfff640 1 --2- 172.21.15.142:0/3303383916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0ce3c0 0x7f11ac0c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.043+0000 7f11abfff640 1 -- 172.21.15.142:0/3303383916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac0cdfe0 msgr2=0x7f11ac0c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.043+0000 7f11abfff640 1 --2- 172.21.15.142:0/3303383916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac0cdfe0 0x7f11ac0c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.043+0000 7f11abfff640 1 -- 172.21.15.142:0/3303383916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0c69b0 con 0x7f11ac0712b0 2024-02-21T23:24:52.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.045+0000 7f11abfff640 1 --2- 172.21.15.142:0/3303383916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac0712b0 0x7f11ac0cd480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f119c00e8c0 tx=0x7f119c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:52.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.049+0000 7f11a8ff9640 1 -- 172.21.15.142:0/3303383916 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c017070 con 0x7f11ac0712b0 2024-02-21T23:24:52.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.049+0000 7f11a8ff9640 1 -- 172.21.15.142:0/3303383916 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c00c940 con 0x7f11ac0712b0 2024-02-21T23:24:52.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.049+0000 7f11b271e640 1 -- 172.21.15.142:0/3303383916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac0c6c90 con 0x7f11ac0712b0 2024-02-21T23:24:52.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.049+0000 7f11a8ff9640 1 -- 172.21.15.142:0/3303383916 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c00cd10 con 0x7f11ac0712b0 2024-02-21T23:24:52.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.049+0000 7f11b271e640 1 -- 172.21.15.142:0/3303383916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac0ccf60 con 0x7f11ac0712b0 2024-02-21T23:24:52.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.050+0000 7f11b271e640 1 -- 172.21.15.142:0/3303383916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ac06a130 con 0x7f11ac0712b0 2024-02-21T23:24:52.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.053+0000 7f11a8ff9640 1 -- 172.21.15.142:0/3303383916 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f119c01a070 con 0x7f11ac0712b0 2024-02-21T23:24:52.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.053+0000 7f11a8ff9640 1 -- 172.21.15.142:0/3303383916 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f119c013470 con 0x7f11ac0712b0 2024-02-21T23:24:52.084 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:52.098+0000 7f0e50021040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-21T23:24:52.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.152+0000 7f11a8ff9640 1 -- 172.21.15.142:0/3303383916 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f119c006490 con 0x7f11ac0712b0 2024-02-21T23:24:52.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.225+0000 7f11b271e640 1 -- 172.21.15.142:0/3303383916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9afcbb30-e40c-4c41-83c2-6886fcb8ee78", "id": 2} v 0) v1 -- 0x7f11ac0c3430 con 0x7f11ac0712b0 2024-02-21T23:24:52.221 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:52.234+0000 7f0e50021040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-21T23:24:52.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.254+0000 7f11a8ff9640 1 -- 172.21.15.142:0/3303383916 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9afcbb30-e40c-4c41-83c2-6886fcb8ee78", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f119c006750 con 0x7f11ac0712b0 2024-02-21T23:24:52.240 INFO:teuthology.orchestra.run.smithi142.stdout:2 2024-02-21T23:24:52.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.255+0000 7f11b271e640 1 -- 172.21.15.142:0/3303383916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac0712b0 msgr2=0x7f11ac0cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.255+0000 7f11b271e640 1 --2- 172.21.15.142:0/3303383916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac0712b0 0x7f11ac0cd480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f119c00e8c0 tx=0x7f119c00ed80 comp rx=0 tx=0).stop 2024-02-21T23:24:52.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.255+0000 7f11b271e640 1 -- 172.21.15.142:0/3303383916 shutdown_connections 2024-02-21T23:24:52.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.255+0000 7f11b271e640 1 --2- 172.21.15.142:0/3303383916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0ce3c0 0x7f11ac0c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.255+0000 7f11b271e640 1 --2- 172.21.15.142:0/3303383916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac0cdfe0 0x7f11ac0c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.255+0000 7f11b271e640 1 --2- 172.21.15.142:0/3303383916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac0712b0 0x7f11ac0cd480 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.255+0000 7f11b271e640 1 -- 172.21.15.142:0/3303383916 >> 172.21.15.142:0/3303383916 conn(0x7f11ac06aec0 msgr2=0x7f11ac14d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:52.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.255+0000 7f11b271e640 1 -- 172.21.15.142:0/3303383916 shutdown_connections 2024-02-21T23:24:52.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.255+0000 7f11b271e640 1 -- 172.21.15.142:0/3303383916 wait complete. 2024-02-21T23:24:52.244 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:52.258+0000 7f8a1d6da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-21T23:24:52.256 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 6d376633-e1d0-415c-86b9-1b94a1775497 3 2024-02-21T23:24:52.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.370+0000 7f3608ca2640 1 Processor -- start 2024-02-21T23:24:52.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.370+0000 7f3608ca2640 1 -- start start 2024-02-21T23:24:52.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.371+0000 7f3608ca2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3604154510 0x7f36040c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.371+0000 7f3608ca2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36040c7a30 con 0x7f360415a6b0 2024-02-21T23:24:52.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.371+0000 7f3608ca2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36040c7cf0 con 0x7f3604072900 2024-02-21T23:24:52.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.371+0000 7f3608ca2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36040c7fd0 con 0x7f3604154510 2024-02-21T23:24:52.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.371+0000 7f360259c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3604154510 0x7f36040c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.372+0000 7f3602d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f360415a6b0 0x7f3604164840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.142:57726/0 (socket says 172.21.15.142:57726) 2024-02-21T23:24:52.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.372+0000 7f3602d9d640 1 -- 172.21.15.142:0/4278012711 learned_addr learned my addr 172.21.15.142:0/4278012711 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-02-21T23:24:52.359 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:52.373+0000 7f0e50021040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-21T23:24:52.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.374+0000 7f360159a640 1 -- 172.21.15.142:0/4278012711 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164411018 0 0) 0x7f36040c7a30 con 0x7f360415a6b0 2024-02-21T23:24:52.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.374+0000 7f360159a640 1 -- 172.21.15.142:0/4278012711 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ec003520 con 0x7f360415a6b0 2024-02-21T23:24:52.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.374+0000 7f360259c640 1 -- 172.21.15.142:0/4278012711 >> v1:172.21.15.142:6789/0 conn(0x7f3604072900 legacy=0x7f3604159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:52.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.374+0000 7f360259c640 1 -- 172.21.15.142:0/4278012711 >> v1:172.21.15.133:6789/0 conn(0x7f360415a6b0 legacy=0x7f3604164840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.374+0000 7f360259c640 1 -- 172.21.15.142:0/4278012711 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36040719f0 con 0x7f3604154510 2024-02-21T23:24:52.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.376+0000 7f360259c640 1 --2- 172.21.15.142:0/4278012711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604154510 0x7f36040c1bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f35f80097a0 tx=0x7f35f802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec833ff887b06c12 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:52.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.381+0000 7f360159a640 1 -- 172.21.15.142:0/4278012711 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8034070 con 0x7f3604154510 2024-02-21T23:24:52.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.381+0000 7f360159a640 1 -- 172.21.15.142:0/4278012711 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f8030610 con 0x7f3604154510 2024-02-21T23:24:52.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.381+0000 7f360159a640 1 -- 172.21.15.142:0/4278012711 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f80309e0 con 0x7f3604154510 2024-02-21T23:24:52.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.382+0000 7f3608ca2640 1 -- 172.21.15.142:0/4278012711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604154510 msgr2=0x7f36040c1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.382+0000 7f3608ca2640 1 --2- 172.21.15.142:0/4278012711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604154510 0x7f36040c1bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f35f80097a0 tx=0x7f35f802b960 comp rx=0 tx=0).stop 2024-02-21T23:24:52.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.382+0000 7f3608ca2640 1 -- 172.21.15.142:0/4278012711 shutdown_connections 2024-02-21T23:24:52.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.382+0000 7f3608ca2640 1 --2- 172.21.15.142:0/4278012711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604154510 0x7f36040c1bf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.382+0000 7f3608ca2640 1 -- 172.21.15.142:0/4278012711 >> 172.21.15.142:0/4278012711 conn(0x7f360406f6e0 msgr2=0x7f360406faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:52.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.382+0000 7f3608ca2640 1 -- 172.21.15.142:0/4278012711 shutdown_connections 2024-02-21T23:24:52.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.382+0000 7f3608ca2640 1 -- 172.21.15.142:0/4278012711 wait complete. 2024-02-21T23:24:52.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.384+0000 7f3608ca2640 1 Processor -- start 2024-02-21T23:24:52.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.384+0000 7f3608ca2640 1 -- start start 2024-02-21T23:24:52.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.384+0000 7f3608ca2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604072900 0x7f3604156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.384+0000 7f3608ca2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3604154510 0x7f360406d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.384+0000 7f3608ca2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f360415a6b0 0x7f36040c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.384+0000 7f3608ca2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3604072f60 con 0x7f3604154510 2024-02-21T23:24:52.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.384+0000 7f3608ca2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3604071fb0 con 0x7f3604072900 2024-02-21T23:24:52.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.384+0000 7f3608ca2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36040725e0 con 0x7f360415a6b0 2024-02-21T23:24:52.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.385+0000 7f360259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604072900 0x7f3604156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.385+0000 7f3602d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f360415a6b0 0x7f36040c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.385+0000 7f3601d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3604154510 0x7f360406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.385+0000 7f3601d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3604154510 0x7f360406d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.142:58822/0 (socket says 172.21.15.142:58822) 2024-02-21T23:24:52.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.385+0000 7f3601d9b640 1 -- 172.21.15.142:0/1984798907 learned_addr learned my addr 172.21.15.142:0/1984798907 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:52.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.389+0000 7f3601d9b640 1 -- 172.21.15.142:0/1984798907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f360415a6b0 msgr2=0x7f36040c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.389+0000 7f3601d9b640 1 --2- 172.21.15.142:0/1984798907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f360415a6b0 0x7f36040c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.389+0000 7f3601d9b640 1 -- 172.21.15.142:0/1984798907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604072900 msgr2=0x7f3604156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.389+0000 7f3601d9b640 1 --2- 172.21.15.142:0/1984798907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604072900 0x7f3604156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.389+0000 7f3601d9b640 1 -- 172.21.15.142:0/1984798907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36040c5090 con 0x7f3604154510 2024-02-21T23:24:52.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.389+0000 7f360259c640 1 --2- 172.21.15.142:0/1984798907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604072900 0x7f3604156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:24:52.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.391+0000 7f3601d9b640 1 --2- 172.21.15.142:0/1984798907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3604154510 0x7f360406d230 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f35f400c440 tx=0x7f35f400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:52.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.394+0000 7f35f2ffd640 1 -- 172.21.15.142:0/1984798907 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4017070 con 0x7f3604154510 2024-02-21T23:24:52.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.394+0000 7f35f2ffd640 1 -- 172.21.15.142:0/1984798907 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f40035f0 con 0x7f3604154510 2024-02-21T23:24:52.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.394+0000 7f35f2ffd640 1 -- 172.21.15.142:0/1984798907 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40039e0 con 0x7f3604154510 2024-02-21T23:24:52.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.394+0000 7f3608ca2640 1 -- 172.21.15.142:0/1984798907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36040c5350 con 0x7f3604154510 2024-02-21T23:24:52.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.395+0000 7f3608ca2640 1 -- 172.21.15.142:0/1984798907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36041577a0 con 0x7f3604154510 2024-02-21T23:24:52.382 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:52.394+0000 7f8a1d6da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-21T23:24:52.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.395+0000 7f3608ca2640 1 -- 172.21.15.142:0/1984798907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360406a130 con 0x7f3604154510 2024-02-21T23:24:52.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.397+0000 7f35f2ffd640 1 -- 172.21.15.142:0/1984798907 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f35f401a070 con 0x7f3604154510 2024-02-21T23:24:52.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.397+0000 7f35f2ffd640 1 -- 172.21.15.142:0/1984798907 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f35f401b840 con 0x7f3604154510 2024-02-21T23:24:52.411 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:52.425+0000 7f0e50021040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-21T23:24:52.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.494+0000 7f35f2ffd640 1 -- 172.21.15.142:0/1984798907 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f35f4021020 con 0x7f3604154510 2024-02-21T23:24:52.509 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:52.523+0000 7f8a1d6da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-21T23:24:52.529 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:52.542+0000 7f0e50021040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-21T23:24:52.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.568+0000 7f3608ca2640 1 -- 172.21.15.142:0/1984798907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6d376633-e1d0-415c-86b9-1b94a1775497", "id": 3} v 0) v1 -- 0x7f36040c7a30 con 0x7f3604154510 2024-02-21T23:24:52.555 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:52.568+0000 7f8a1d6da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-21T23:24:52.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.621+0000 7f35f2ffd640 1 -- 172.21.15.142:0/1984798907 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6d376633-e1d0-415c-86b9-1b94a1775497", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f35f4003c50 con 0x7f3604154510 2024-02-21T23:24:52.608 INFO:teuthology.orchestra.run.smithi142.stdout:3 2024-02-21T23:24:52.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.622+0000 7f3608ca2640 1 -- 172.21.15.142:0/1984798907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3604154510 msgr2=0x7f360406d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.622+0000 7f3608ca2640 1 --2- 172.21.15.142:0/1984798907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3604154510 0x7f360406d230 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f35f400c440 tx=0x7f35f400c900 comp rx=0 tx=0).stop 2024-02-21T23:24:52.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.622+0000 7f3608ca2640 1 -- 172.21.15.142:0/1984798907 shutdown_connections 2024-02-21T23:24:52.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.622+0000 7f3608ca2640 1 --2- 172.21.15.142:0/1984798907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f360415a6b0 0x7f36040c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.622+0000 7f3608ca2640 1 --2- 172.21.15.142:0/1984798907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3604154510 0x7f360406d230 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.622+0000 7f3608ca2640 1 --2- 172.21.15.142:0/1984798907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3604072900 0x7f3604156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.622+0000 7f3608ca2640 1 -- 172.21.15.142:0/1984798907 >> 172.21.15.142:0/1984798907 conn(0x7f360406f6e0 msgr2=0x7f360406fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:52.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.622+0000 7f3608ca2640 1 -- 172.21.15.142:0/1984798907 shutdown_connections 2024-02-21T23:24:52.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.622+0000 7f3608ca2640 1 -- 172.21.15.142:0/1984798907 wait complete. 2024-02-21T23:24:52.620 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new a47be052-7790-4954-af9b-20ec83394f00 4 2024-02-21T23:24:52.645 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:52.658+0000 7f0e50021040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-21T23:24:52.672 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:52.685+0000 7f8a1d6da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-21T23:24:52.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.735+0000 7f2d12daa640 1 Processor -- start 2024-02-21T23:24:52.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.735+0000 7f2d12daa640 1 -- start start 2024-02-21T23:24:52.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.735+0000 7f2d12daa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 0x7f2d0c06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.735+0000 7f2d12daa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2d0c1587b0 0x7f2d0c151960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.735+0000 7f2d12daa640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c070b40 con 0x7f2d0c0c63b0 2024-02-21T23:24:52.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.735+0000 7f2d12daa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c0cd460 con 0x7f2d0c1587b0 2024-02-21T23:24:52.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.735+0000 7f2d12daa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0c7af0 con 0x7f2d0c0c5fd0 2024-02-21T23:24:52.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.736+0000 7f2d10b46640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 0x7f2d0c06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.736+0000 7f2d11347640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2d0c1587b0 0x7f2d0c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.736+0000 7f2d10b46640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 0x7f2d0c06b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:39964/0 (socket says 172.21.15.142:39964) 2024-02-21T23:24:52.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.736+0000 7f2d10b46640 1 -- 172.21.15.142:0/73982962 learned_addr learned my addr 172.21.15.142:0/73982962 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:52.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.739+0000 7f2d10b46640 1 -- 172.21.15.142:0/73982962 >> v2:172.21.15.142:3300/0 conn(0x7f2d0c1587b0 msgr2=0x7f2d0c151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:52.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.739+0000 7f2d10b46640 1 --2- 172.21.15.142:0/73982962 >> v2:172.21.15.142:3300/0 conn(0x7f2d0c1587b0 0x7f2d0c151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.739+0000 7f2d10b46640 1 -- 172.21.15.142:0/73982962 >> v1:172.21.15.142:6789/0 conn(0x7f2d0c0c63b0 legacy=0x7f2d0c1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:52.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.739+0000 7f2d10b46640 1 -- 172.21.15.142:0/73982962 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c070860 con 0x7f2d0c0c5fd0 2024-02-21T23:24:52.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.741+0000 7f2d10b46640 1 --2- 172.21.15.142:0/73982962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 0x7f2d0c06b7a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009870 tx=0x7f2d0002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3b37cb06ab3741a server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:52.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.746+0000 7f2d0b7fe640 1 -- 172.21.15.142:0/73982962 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00034070 con 0x7f2d0c0c5fd0 2024-02-21T23:24:52.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.746+0000 7f2d0b7fe640 1 -- 172.21.15.142:0/73982962 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d00030840 con 0x7f2d0c0c5fd0 2024-02-21T23:24:52.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.746+0000 7f2d0b7fe640 1 -- 172.21.15.142:0/73982962 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00030c10 con 0x7f2d0c0c5fd0 2024-02-21T23:24:52.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.746+0000 7f2d12daa640 1 -- 172.21.15.142:0/73982962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 msgr2=0x7f2d0c06b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.746+0000 7f2d12daa640 1 --2- 172.21.15.142:0/73982962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 0x7f2d0c06b7a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009870 tx=0x7f2d0002b980 comp rx=0 tx=0).stop 2024-02-21T23:24:52.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.747+0000 7f2d12daa640 1 -- 172.21.15.142:0/73982962 shutdown_connections 2024-02-21T23:24:52.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.747+0000 7f2d12daa640 1 --2- 172.21.15.142:0/73982962 >> v2:172.21.15.142:3300/0 conn(0x7f2d0c1587b0 0x7f2d0c151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.747+0000 7f2d12daa640 1 --2- 172.21.15.142:0/73982962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 0x7f2d0c06b7a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.747+0000 7f2d12daa640 1 -- 172.21.15.142:0/73982962 >> 172.21.15.142:0/73982962 conn(0x7f2d0c061b50 msgr2=0x7f2d0c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:52.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.747+0000 7f2d12daa640 1 -- 172.21.15.142:0/73982962 shutdown_connections 2024-02-21T23:24:52.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.747+0000 7f2d12daa640 1 -- 172.21.15.142:0/73982962 wait complete. 2024-02-21T23:24:52.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.748+0000 7f2d12daa640 1 Processor -- start 2024-02-21T23:24:52.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.748+0000 7f2d12daa640 1 -- start start 2024-02-21T23:24:52.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.749+0000 7f2d12daa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 0x7f2d0c06c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.749+0000 7f2d12daa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c0c63b0 0x7f2d0c0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.749+0000 7f2d12daa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c1587b0 0x7f2d0c1ee250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:52.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.749+0000 7f2d12daa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0cd0f0 con 0x7f2d0c1587b0 2024-02-21T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.749+0000 7f2d12daa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0cce10 con 0x7f2d0c0c63b0 2024-02-21T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.749+0000 7f2d12daa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c06bce0 con 0x7f2d0c0c5fd0 2024-02-21T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.749+0000 7f2d11347640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c1587b0 0x7f2d0c1ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.749+0000 7f2d11347640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c1587b0 0x7f2d0c1ee250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.142:58828/0 (socket says 172.21.15.142:58828) 2024-02-21T23:24:52.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.749+0000 7f2d11347640 1 -- 172.21.15.142:0/655671792 learned_addr learned my addr 172.21.15.142:0/655671792 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:52.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.751+0000 7f2d0bfff640 1 --2- 172.21.15.142:0/655671792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c0c63b0 0x7f2d0c0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.753+0000 7f2d11347640 1 -- 172.21.15.142:0/655671792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 msgr2=0x7f2d0c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.753+0000 7f2d11347640 1 --2- 172.21.15.142:0/655671792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 0x7f2d0c06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.753+0000 7f2d11347640 1 -- 172.21.15.142:0/655671792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c0c63b0 msgr2=0x7f2d0c0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.753+0000 7f2d11347640 1 --2- 172.21.15.142:0/655671792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c0c63b0 0x7f2d0c0cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.753+0000 7f2d11347640 1 -- 172.21.15.142:0/655671792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c1f14c0 con 0x7f2d0c1587b0 2024-02-21T23:24:52.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.755+0000 7f2d11347640 1 --2- 172.21.15.142:0/655671792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c1587b0 0x7f2d0c1ee250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400efc0 tx=0x7f2d0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:52.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.766+0000 7f2d097fa640 1 -- 172.21.15.142:0/655671792 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04018020 con 0x7f2d0c1587b0 2024-02-21T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.766+0000 7f2d097fa640 1 -- 172.21.15.142:0/655671792 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04014480 con 0x7f2d0c1587b0 2024-02-21T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.766+0000 7f2d12daa640 1 -- 172.21.15.142:0/655671792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c1f17a0 con 0x7f2d0c1587b0 2024-02-21T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.766+0000 7f2d097fa640 1 -- 172.21.15.142:0/655671792 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04014870 con 0x7f2d0c1587b0 2024-02-21T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.766+0000 7f2d12daa640 1 -- 172.21.15.142:0/655671792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c1f1d60 con 0x7f2d0c1587b0 2024-02-21T23:24:52.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.767+0000 7f2d12daa640 1 -- 172.21.15.142:0/655671792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c06a130 con 0x7f2d0c1587b0 2024-02-21T23:24:52.770 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:52.783+0000 7f8a1d6da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-21T23:24:52.828 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:52.842+0000 7f0e50021040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-21T23:24:52.870 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:52.884+0000 7f0e50021040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-21T23:24:52.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.899+0000 7f2d097fa640 1 -- 172.21.15.142:0/655671792 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2d04005d10 con 0x7f2d0c1587b0 2024-02-21T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.903+0000 7f2d097fa640 1 -- 172.21.15.142:0/655671792 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2d040138f0 con 0x7f2d0c1587b0 2024-02-21T23:24:52.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.903+0000 7f2d097fa640 1 -- 172.21.15.142:0/655671792 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2d04022070 con 0x7f2d0c1587b0 2024-02-21T23:24:52.927 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:52.940+0000 7f8a1d6da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-21T23:24:52.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:52.976+0000 7f2d12daa640 1 -- 172.21.15.142:0/655671792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a47be052-7790-4954-af9b-20ec83394f00", "id": 4} v 0) v1 -- 0x7f2d0c0686d0 con 0x7f2d0c1587b0 2024-02-21T23:24:52.969 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:52.982+0000 7f8a1d6da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-21T23:24:52.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.008+0000 7f2d097fa640 1 -- 172.21.15.142:0/655671792 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a47be052-7790-4954-af9b-20ec83394f00", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2d04010040 con 0x7f2d0c1587b0 2024-02-21T23:24:52.994 INFO:teuthology.orchestra.run.smithi142.stdout:4 2024-02-21T23:24:52.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.009+0000 7f2d12daa640 1 -- 172.21.15.142:0/655671792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c1587b0 msgr2=0x7f2d0c1ee250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.009+0000 7f2d12daa640 1 --2- 172.21.15.142:0/655671792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c1587b0 0x7f2d0c1ee250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400efc0 tx=0x7f2d0400c490 comp rx=0 tx=0).stop 2024-02-21T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.009+0000 7f2d12daa640 1 -- 172.21.15.142:0/655671792 shutdown_connections 2024-02-21T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.009+0000 7f2d12daa640 1 --2- 172.21.15.142:0/655671792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c1587b0 0x7f2d0c1ee250 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.009+0000 7f2d12daa640 1 --2- 172.21.15.142:0/655671792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c0c63b0 0x7f2d0c0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.009+0000 7f2d12daa640 1 --2- 172.21.15.142:0/655671792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d0c0c5fd0 0x7f2d0c06c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.009+0000 7f2d12daa640 1 -- 172.21.15.142:0/655671792 >> 172.21.15.142:0/655671792 conn(0x7f2d0c061b50 msgr2=0x7f2d0c150e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.009+0000 7f2d12daa640 1 -- 172.21.15.142:0/655671792 shutdown_connections 2024-02-21T23:24:52.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.009+0000 7f2d12daa640 1 -- 172.21.15.142:0/655671792 wait complete. 2024-02-21T23:24:53.007 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new bf00fa69-437a-4077-8ef5-629955908024 5 2024-02-21T23:24:53.046 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:53.060+0000 7f0e50021040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-21T23:24:53.127 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:53.140+0000 7f8a1d6da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-21T23:24:53.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.160+0000 7fb841a60640 1 Processor -- start 2024-02-21T23:24:53.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.160+0000 7fb841a60640 1 -- start start 2024-02-21T23:24:53.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.161+0000 7fb841a60640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c154600 0x7fb83c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.161+0000 7fb841a60640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb83c0c6f50 0x7fb83c0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.161+0000 7fb841a60640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb83c0cab80 0x7fb83c0c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.161+0000 7fb841a60640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb83c0c74a0 con 0x7fb83c0c6f50 2024-02-21T23:24:53.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.161+0000 7fb841a60640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb83c0c7780 con 0x7fb83c0cab80 2024-02-21T23:24:53.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.161+0000 7fb841a60640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c15ab40 con 0x7fb83c154600 2024-02-21T23:24:53.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.162+0000 7fb83affd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c154600 0x7fb83c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.162+0000 7fb83b7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb83c0cab80 0x7fb83c0c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.162+0000 7fb83a7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb83c0c6f50 0x7fb83c0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.162+0000 7fb83a7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb83c0c6f50 0x7fb83c0ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.142:58830/0 (socket says 172.21.15.142:58830) 2024-02-21T23:24:53.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.162+0000 7fb83a7fc640 1 -- 172.21.15.142:0/1962909521 learned_addr learned my addr 172.21.15.142:0/1962909521 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:53.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.164+0000 7fb83a7fc640 1 -- 172.21.15.142:0/1962909521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c154600 msgr2=0x7fb83c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:53.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.164+0000 7fb83a7fc640 1 --2- 172.21.15.142:0/1962909521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c154600 0x7fb83c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.164+0000 7fb83a7fc640 1 -- 172.21.15.142:0/1962909521 >> v2:172.21.15.142:3300/0 conn(0x7fb83c0cab80 msgr2=0x7fb83c0c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:53.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.164+0000 7fb83a7fc640 1 --2- 172.21.15.142:0/1962909521 >> v2:172.21.15.142:3300/0 conn(0x7fb83c0cab80 0x7fb83c0c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.164+0000 7fb83a7fc640 1 -- 172.21.15.142:0/1962909521 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c15ae00 con 0x7fb83c0c6f50 2024-02-21T23:24:53.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.166+0000 7fb83a7fc640 1 --2- 172.21.15.142:0/1962909521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c0c6f50 0x7fb83c0ca640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0c6700 tx=0x7fb82c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6f86dbca40f502 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:53.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb839ffb640 1 -- 172.21.15.142:0/1962909521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c035020 con 0x7fb83c0c6f50 2024-02-21T23:24:53.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb839ffb640 1 -- 172.21.15.142:0/1962909521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c031870 con 0x7fb83c0c6f50 2024-02-21T23:24:53.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb839ffb640 1 -- 172.21.15.142:0/1962909521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c031c40 con 0x7fb83c0c6f50 2024-02-21T23:24:53.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb841a60640 1 -- 172.21.15.142:0/1962909521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c0c6f50 msgr2=0x7fb83c0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb841a60640 1 --2- 172.21.15.142:0/1962909521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c0c6f50 0x7fb83c0ca640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0c6700 tx=0x7fb82c02b910 comp rx=0 tx=0).stop 2024-02-21T23:24:53.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb841a60640 1 -- 172.21.15.142:0/1962909521 shutdown_connections 2024-02-21T23:24:53.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb841a60640 1 --2- 172.21.15.142:0/1962909521 >> v2:172.21.15.142:3300/0 conn(0x7fb83c0cab80 0x7fb83c0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb841a60640 1 --2- 172.21.15.142:0/1962909521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c0c6f50 0x7fb83c0ca640 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb841a60640 1 --2- 172.21.15.142:0/1962909521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c154600 0x7fb83c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb841a60640 1 -- 172.21.15.142:0/1962909521 >> 172.21.15.142:0/1962909521 conn(0x7fb83c062000 msgr2=0x7fb83c0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:53.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.170+0000 7fb841a60640 1 -- 172.21.15.142:0/1962909521 shutdown_connections 2024-02-21T23:24:53.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.171+0000 7fb841a60640 1 -- 172.21.15.142:0/1962909521 wait complete. 2024-02-21T23:24:53.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.172+0000 7fb841a60640 1 Processor -- start 2024-02-21T23:24:53.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.172+0000 7fb841a60640 1 -- start start 2024-02-21T23:24:53.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.172+0000 7fb841a60640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c0c6f50 0x7fb83c14ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.172+0000 7fb841a60640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb83c0cab80 0x7fb83c155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.172+0000 7fb841a60640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c154600 0x7fb83c155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.172+0000 7fb841a60640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c0b5720 con 0x7fb83c154600 2024-02-21T23:24:53.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.172+0000 7fb841a60640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c0cb860 con 0x7fb83c0cab80 2024-02-21T23:24:53.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.172+0000 7fb841a60640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c0b58a0 con 0x7fb83c0c6f50 2024-02-21T23:24:53.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.173+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c154600 0x7fb83c155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.173+0000 7fb83a7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb83c0cab80 0x7fb83c155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.173+0000 7fb83affd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c0c6f50 0x7fb83c14ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.173+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c154600 0x7fb83c155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.142:58846/0 (socket says 172.21.15.142:58846) 2024-02-21T23:24:53.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.173+0000 7fb83b7fe640 1 -- 172.21.15.142:0/1023123134 learned_addr learned my addr 172.21.15.142:0/1023123134 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:53.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.177+0000 7fb83b7fe640 1 -- 172.21.15.142:0/1023123134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c0c6f50 msgr2=0x7fb83c14ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.177+0000 7fb83b7fe640 1 --2- 172.21.15.142:0/1023123134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c0c6f50 0x7fb83c14ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.177+0000 7fb83b7fe640 1 -- 172.21.15.142:0/1023123134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb83c0cab80 msgr2=0x7fb83c155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.177+0000 7fb83b7fe640 1 --2- 172.21.15.142:0/1023123134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb83c0cab80 0x7fb83c155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.177+0000 7fb83b7fe640 1 -- 172.21.15.142:0/1023123134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c1f38a0 con 0x7fb83c154600 2024-02-21T23:24:53.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.179+0000 7fb83b7fe640 1 --2- 172.21.15.142:0/1023123134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c154600 0x7fb83c155950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb83400ef00 tx=0x7fb83400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:53.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.182+0000 7fb823fff640 1 -- 172.21.15.142:0/1023123134 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834018020 con 0x7fb83c154600 2024-02-21T23:24:53.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.182+0000 7fb823fff640 1 -- 172.21.15.142:0/1023123134 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb834014400 con 0x7fb83c154600 2024-02-21T23:24:53.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.182+0000 7fb823fff640 1 -- 172.21.15.142:0/1023123134 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8340147f0 con 0x7fb83c154600 2024-02-21T23:24:53.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.182+0000 7fb841a60640 1 -- 172.21.15.142:0/1023123134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83c1f3b80 con 0x7fb83c154600 2024-02-21T23:24:53.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.182+0000 7fb841a60640 1 -- 172.21.15.142:0/1023123134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c1f4140 con 0x7fb83c154600 2024-02-21T23:24:53.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.183+0000 7fb841a60640 1 -- 172.21.15.142:0/1023123134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83c1f4450 con 0x7fb83c154600 2024-02-21T23:24:53.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.185+0000 7fb823fff640 1 -- 172.21.15.142:0/1023123134 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb834005d10 con 0x7fb83c154600 2024-02-21T23:24:53.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.185+0000 7fb823fff640 1 -- 172.21.15.142:0/1023123134 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fb8340253d0 con 0x7fb83c154600 2024-02-21T23:24:53.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.281+0000 7fb823fff640 1 -- 172.21.15.142:0/1023123134 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb834022070 con 0x7fb83c154600 2024-02-21T23:24:53.339 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.353+0000 7fb841a60640 1 -- 172.21.15.142:0/1023123134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bf00fa69-437a-4077-8ef5-629955908024", "id": 5} v 0) v1 -- 0x7fb83c06a130 con 0x7fb83c154600 2024-02-21T23:24:53.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.384+0000 7fb823fff640 1 -- 172.21.15.142:0/1023123134 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bf00fa69-437a-4077-8ef5-629955908024", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb834026420 con 0x7fb83c154600 2024-02-21T23:24:53.371 INFO:teuthology.orchestra.run.smithi142.stdout:5 2024-02-21T23:24:53.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.385+0000 7fb841a60640 1 -- 172.21.15.142:0/1023123134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c154600 msgr2=0x7fb83c155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.385+0000 7fb841a60640 1 --2- 172.21.15.142:0/1023123134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c154600 0x7fb83c155950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb83400ef00 tx=0x7fb83400b480 comp rx=0 tx=0).stop 2024-02-21T23:24:53.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.385+0000 7fb841a60640 1 -- 172.21.15.142:0/1023123134 shutdown_connections 2024-02-21T23:24:53.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.385+0000 7fb841a60640 1 --2- 172.21.15.142:0/1023123134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83c154600 0x7fb83c155950 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.385+0000 7fb841a60640 1 --2- 172.21.15.142:0/1023123134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb83c0cab80 0x7fb83c155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.385+0000 7fb841a60640 1 --2- 172.21.15.142:0/1023123134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb83c0c6f50 0x7fb83c14ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.385+0000 7fb841a60640 1 -- 172.21.15.142:0/1023123134 >> 172.21.15.142:0/1023123134 conn(0x7fb83c062000 msgr2=0x7fb83c0c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:53.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.385+0000 7fb841a60640 1 -- 172.21.15.142:0/1023123134 shutdown_connections 2024-02-21T23:24:53.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.385+0000 7fb841a60640 1 -- 172.21.15.142:0/1023123134 wait complete. 2024-02-21T23:24:53.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 9ce52b5c-d828-40a1-80bf-3456f148f2a1 6 2024-02-21T23:24:53.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.501+0000 7f062efdc640 1 Processor -- start 2024-02-21T23:24:53.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.501+0000 7f062efdc640 1 -- start start 2024-02-21T23:24:53.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.501+0000 7f062efdc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0628072900 0x7f062815d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.501+0000 7f062efdc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f062815da50 0x7f062815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.501+0000 7f062efdc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06280c2300 con 0x7f0628154510 2024-02-21T23:24:53.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.502+0000 7f062efdc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06280c7fd0 con 0x7f0628072900 2024-02-21T23:24:53.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.502+0000 7f062efdc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0628071fb0 con 0x7f062815da50 2024-02-21T23:24:53.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.502+0000 7f062d579640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f062815da50 0x7f062815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.502+0000 7f062cd78640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f0628154510 0x7f06280c1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:34352/0 (socket says 172.21.15.142:34352) 2024-02-21T23:24:53.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.502+0000 7f062cd78640 1 -- 172.21.15.142:0/4282005175 learned_addr learned my addr 172.21.15.142:0/4282005175 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-02-21T23:24:53.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.502+0000 7f0627fff640 1 --2- 172.21.15.142:0/4282005175 >> v2:172.21.15.133:3300/0 conn(0x7f0628072900 0x7f062815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.504+0000 7f0627fff640 1 -- 172.21.15.142:0/4282005175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f062815da50 msgr2=0x7f062815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:53.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.504+0000 7f0627fff640 1 --2- 172.21.15.142:0/4282005175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f062815da50 0x7f062815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.504+0000 7f0627fff640 1 -- 172.21.15.142:0/4282005175 >> v1:172.21.15.142:6789/0 conn(0x7f0628154510 legacy=0x7f06280c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.504+0000 7f0627fff640 1 -- 172.21.15.142:0/4282005175 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062815e420 con 0x7f0628072900 2024-02-21T23:24:53.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.506+0000 7f0627fff640 1 --2- 172.21.15.142:0/4282005175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628072900 0x7f062815d510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06140097a0 tx=0x7f061402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b6060bf9b84373 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:53.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.510+0000 7f06277fe640 1 -- 172.21.15.142:0/4282005175 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614034070 con 0x7f0628072900 2024-02-21T23:24:53.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.510+0000 7f06277fe640 1 -- 172.21.15.142:0/4282005175 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614030610 con 0x7f0628072900 2024-02-21T23:24:53.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.510+0000 7f06277fe640 1 -- 172.21.15.142:0/4282005175 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140309e0 con 0x7f0628072900 2024-02-21T23:24:53.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.511+0000 7f062efdc640 1 -- 172.21.15.142:0/4282005175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628072900 msgr2=0x7f062815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.511+0000 7f062efdc640 1 --2- 172.21.15.142:0/4282005175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628072900 0x7f062815d510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06140097a0 tx=0x7f061402b960 comp rx=0 tx=0).stop 2024-02-21T23:24:53.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.511+0000 7f062efdc640 1 -- 172.21.15.142:0/4282005175 shutdown_connections 2024-02-21T23:24:53.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.511+0000 7f062efdc640 1 --2- 172.21.15.142:0/4282005175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f062815da50 0x7f062815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.511+0000 7f062efdc640 1 --2- 172.21.15.142:0/4282005175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628072900 0x7f062815d510 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.511+0000 7f062efdc640 1 -- 172.21.15.142:0/4282005175 >> 172.21.15.142:0/4282005175 conn(0x7f062806f6e0 msgr2=0x7f062806faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:53.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.511+0000 7f062efdc640 1 -- 172.21.15.142:0/4282005175 shutdown_connections 2024-02-21T23:24:53.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.511+0000 7f062efdc640 1 -- 172.21.15.142:0/4282005175 wait complete. 2024-02-21T23:24:53.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.512+0000 7f062efdc640 1 Processor -- start 2024-02-21T23:24:53.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.513+0000 7f062efdc640 1 -- start start 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.513+0000 7f062efdc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0628072900 0x7f06281534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.513+0000 7f062efdc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628154510 0x7f0628153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.513+0000 7f062efdc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062815da50 0x7f06281e5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.513+0000 7f062efdc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06280719f0 con 0x7f0628154510 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.514+0000 7f062efdc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06280c7cf0 con 0x7f062815da50 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.514+0000 7f062efdc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0628071cd0 con 0x7f0628072900 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.514+0000 7f0627fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628154510 0x7f0628153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.514+0000 7f062d579640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062815da50 0x7f06281e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.514+0000 7f062cd78640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0628072900 0x7f06281534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.514+0000 7f0627fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628154510 0x7f0628153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.142:58858/0 (socket says 172.21.15.142:58858) 2024-02-21T23:24:53.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.514+0000 7f0627fff640 1 -- 172.21.15.142:0/3880480373 learned_addr learned my addr 172.21.15.142:0/3880480373 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:53.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.518+0000 7f0627fff640 1 -- 172.21.15.142:0/3880480373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0628072900 msgr2=0x7f06281534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.518+0000 7f0627fff640 1 --2- 172.21.15.142:0/3880480373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0628072900 0x7f06281534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.518+0000 7f0627fff640 1 -- 172.21.15.142:0/3880480373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062815da50 msgr2=0x7f06281e5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.518+0000 7f0627fff640 1 --2- 172.21.15.142:0/3880480373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062815da50 0x7f06281e5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.518+0000 7f0627fff640 1 -- 172.21.15.142:0/3880480373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06281589f0 con 0x7f0628154510 2024-02-21T23:24:53.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.520+0000 7f0627fff640 1 --2- 172.21.15.142:0/3880480373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628154510 0x7f0628153a00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f06140098c0 tx=0x7f06140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:53.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.523+0000 7f06257fa640 1 -- 172.21.15.142:0/3880480373 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614034040 con 0x7f0628154510 2024-02-21T23:24:53.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.523+0000 7f06257fa640 1 -- 172.21.15.142:0/3880480373 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614030410 con 0x7f0628154510 2024-02-21T23:24:53.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.523+0000 7f062efdc640 1 -- 172.21.15.142:0/3880480373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0628157eb0 con 0x7f0628154510 2024-02-21T23:24:53.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.523+0000 7f06257fa640 1 -- 172.21.15.142:0/3880480373 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614039c70 con 0x7f0628154510 2024-02-21T23:24:53.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.523+0000 7f062efdc640 1 -- 172.21.15.142:0/3880480373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0628158150 con 0x7f0628154510 2024-02-21T23:24:53.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.524+0000 7f062efdc640 1 -- 172.21.15.142:0/3880480373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062806a130 con 0x7f0628154510 2024-02-21T23:24:53.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.526+0000 7f06257fa640 1 -- 172.21.15.142:0/3880480373 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0614037070 con 0x7f0628154510 2024-02-21T23:24:53.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.526+0000 7f06257fa640 1 -- 172.21.15.142:0/3880480373 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f0614055af0 con 0x7f0628154510 2024-02-21T23:24:53.610 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.624+0000 7f06257fa640 1 -- 172.21.15.142:0/3880480373 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f062815e420 con 0x7f0628154510 2024-02-21T23:24:53.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.698+0000 7f062efdc640 1 -- 172.21.15.142:0/3880480373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ce52b5c-d828-40a1-80bf-3456f148f2a1", "id": 6} v 0) v1 -- 0x7f062806df80 con 0x7f0628154510 2024-02-21T23:24:53.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.727+0000 7f06257fa640 1 -- 172.21.15.142:0/3880480373 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ce52b5c-d828-40a1-80bf-3456f148f2a1", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f06140305c0 con 0x7f0628154510 2024-02-21T23:24:53.714 INFO:teuthology.orchestra.run.smithi142.stdout:6 2024-02-21T23:24:53.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.728+0000 7f062efdc640 1 -- 172.21.15.142:0/3880480373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628154510 msgr2=0x7f0628153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.728+0000 7f062efdc640 1 --2- 172.21.15.142:0/3880480373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628154510 0x7f0628153a00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f06140098c0 tx=0x7f06140046c0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.728+0000 7f062efdc640 1 -- 172.21.15.142:0/3880480373 shutdown_connections 2024-02-21T23:24:53.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.728+0000 7f062efdc640 1 --2- 172.21.15.142:0/3880480373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062815da50 0x7f06281e5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.728+0000 7f062efdc640 1 --2- 172.21.15.142:0/3880480373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0628154510 0x7f0628153a00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.728+0000 7f062efdc640 1 --2- 172.21.15.142:0/3880480373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0628072900 0x7f06281534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.728+0000 7f062efdc640 1 -- 172.21.15.142:0/3880480373 >> 172.21.15.142:0/3880480373 conn(0x7f062806f6e0 msgr2=0x7f062815b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:53.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.728+0000 7f062efdc640 1 -- 172.21.15.142:0/3880480373 shutdown_connections 2024-02-21T23:24:53.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.728+0000 7f062efdc640 1 -- 172.21.15.142:0/3880480373 wait complete. 2024-02-21T23:24:53.726 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new a3e6bc6d-31fa-4304-ae53-07f0bfa88924 7 2024-02-21T23:24:53.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.845+0000 7f68effaa640 1 Processor -- start 2024-02-21T23:24:53.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.845+0000 7f68effaa640 1 -- start start 2024-02-21T23:24:53.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.846+0000 7f68effaa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c3dc0 0x7f68e80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.846+0000 7f68effaa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f68e80c41a0 0x7f68e8070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.846+0000 7f68effaa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e8156e90 con 0x7f68e8150d30 2024-02-21T23:24:53.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.846+0000 7f68effaa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e8157150 con 0x7f68e80c41a0 2024-02-21T23:24:53.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.846+0000 7f68effaa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8157430 con 0x7f68e80c3dc0 2024-02-21T23:24:53.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.846+0000 7f68ed545640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f68e80c41a0 0x7f68e8070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.846+0000 7f68edd46640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c3dc0 0x7f68e80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.847+0000 7f68ed545640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f68e80c41a0 0x7f68e8070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.142:58872/0 (socket says 172.21.15.142:58872) 2024-02-21T23:24:53.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.847+0000 7f68ed545640 1 -- 172.21.15.142:0/2741464701 learned_addr learned my addr 172.21.15.142:0/2741464701 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:53.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.849+0000 7f68edd46640 1 -- 172.21.15.142:0/2741464701 >> v2:172.21.15.133:3300/0 conn(0x7f68e80c41a0 msgr2=0x7f68e8070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:24:53.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.849+0000 7f68edd46640 1 --2- 172.21.15.142:0/2741464701 >> v2:172.21.15.133:3300/0 conn(0x7f68e80c41a0 0x7f68e8070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.849+0000 7f68edd46640 1 -- 172.21.15.142:0/2741464701 >> v1:172.21.15.133:6789/0 conn(0x7f68e8150d30 legacy=0x7f68e815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.849+0000 7f68edd46640 1 -- 172.21.15.142:0/2741464701 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8157710 con 0x7f68e80c3dc0 2024-02-21T23:24:53.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.851+0000 7f68edd46640 1 --2- 172.21.15.142:0/2741464701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c3dc0 0x7f68e80cd010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68e40097a0 tx=0x7f68e402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617da8827b8f2978 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:53.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.855+0000 7f68ecd44640 1 -- 172.21.15.142:0/2741464701 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4034070 con 0x7f68e80c3dc0 2024-02-21T23:24:53.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.855+0000 7f68ecd44640 1 -- 172.21.15.142:0/2741464701 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e4030610 con 0x7f68e80c3dc0 2024-02-21T23:24:53.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.855+0000 7f68ecd44640 1 -- 172.21.15.142:0/2741464701 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e40309e0 con 0x7f68e80c3dc0 2024-02-21T23:24:53.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.855+0000 7f68effaa640 1 -- 172.21.15.142:0/2741464701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c3dc0 msgr2=0x7f68e80cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.855+0000 7f68effaa640 1 --2- 172.21.15.142:0/2741464701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c3dc0 0x7f68e80cd010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68e40097a0 tx=0x7f68e402b960 comp rx=0 tx=0).stop 2024-02-21T23:24:53.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.856+0000 7f68effaa640 1 -- 172.21.15.142:0/2741464701 shutdown_connections 2024-02-21T23:24:53.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.856+0000 7f68effaa640 1 --2- 172.21.15.142:0/2741464701 >> v2:172.21.15.133:3300/0 conn(0x7f68e80c41a0 0x7f68e8070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.856+0000 7f68effaa640 1 --2- 172.21.15.142:0/2741464701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c3dc0 0x7f68e80cd010 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.856+0000 7f68effaa640 1 -- 172.21.15.142:0/2741464701 >> 172.21.15.142:0/2741464701 conn(0x7f68e8061b50 msgr2=0x7f68e8072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:53.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.856+0000 7f68effaa640 1 -- 172.21.15.142:0/2741464701 shutdown_connections 2024-02-21T23:24:53.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.856+0000 7f68effaa640 1 -- 172.21.15.142:0/2741464701 wait complete. 2024-02-21T23:24:53.843 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.857+0000 7f68effaa640 1 Processor -- start 2024-02-21T23:24:53.843 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.857+0000 7f68effaa640 1 -- start start 2024-02-21T23:24:53.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.858+0000 7f68effaa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e80c3dc0 0x7f68e80c3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.858+0000 7f68effaa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c41a0 0x7f68e80cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.858+0000 7f68effaa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68e8150d30 0x7f68e813fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:53.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.858+0000 7f68effaa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8070190 con 0x7f68e80c3dc0 2024-02-21T23:24:53.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.858+0000 7f68effaa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8157cd0 con 0x7f68e8150d30 2024-02-21T23:24:53.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.858+0000 7f68effaa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8070950 con 0x7f68e80c41a0 2024-02-21T23:24:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.858+0000 7f68edd46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e80c3dc0 0x7f68e80c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.858+0000 7f68ee547640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68e8150d30 0x7f68e813fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.858+0000 7f68ed545640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c41a0 0x7f68e80cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.859+0000 7f68edd46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e80c3dc0 0x7f68e80c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.142:58882/0 (socket says 172.21.15.142:58882) 2024-02-21T23:24:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.859+0000 7f68edd46640 1 -- 172.21.15.142:0/898935517 learned_addr learned my addr 172.21.15.142:0/898935517 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-21T23:24:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.862+0000 7f68edd46640 1 -- 172.21.15.142:0/898935517 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c41a0 msgr2=0x7f68e80cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.862+0000 7f68edd46640 1 --2- 172.21.15.142:0/898935517 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c41a0 0x7f68e80cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.862+0000 7f68edd46640 1 -- 172.21.15.142:0/898935517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68e8150d30 msgr2=0x7f68e813fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.862+0000 7f68edd46640 1 --2- 172.21.15.142:0/898935517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68e8150d30 0x7f68e813fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.862+0000 7f68edd46640 1 -- 172.21.15.142:0/898935517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8148290 con 0x7f68e80c3dc0 2024-02-21T23:24:53.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.864+0000 7f68edd46640 1 --2- 172.21.15.142:0/898935517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e80c3dc0 0x7f68e80c3170 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f68e40098c0 tx=0x7f68e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:53.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.867+0000 7f68de7fc640 1 -- 172.21.15.142:0/898935517 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4034040 con 0x7f68e80c3dc0 2024-02-21T23:24:53.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.867+0000 7f68de7fc640 1 -- 172.21.15.142:0/898935517 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e403aaa0 con 0x7f68e80c3dc0 2024-02-21T23:24:53.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.867+0000 7f68de7fc640 1 -- 172.21.15.142:0/898935517 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e40443f0 con 0x7f68e80c3dc0 2024-02-21T23:24:53.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.867+0000 7f68effaa640 1 -- 172.21.15.142:0/898935517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e81641e0 con 0x7f68e80c3dc0 2024-02-21T23:24:53.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.867+0000 7f68effaa640 1 -- 172.21.15.142:0/898935517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e81f3f20 con 0x7f68e80c3dc0 2024-02-21T23:24:53.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.869+0000 7f68effaa640 1 -- 172.21.15.142:0/898935517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e806a130 con 0x7f68e80c3dc0 2024-02-21T23:24:53.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.870+0000 7f68de7fc640 1 -- 172.21.15.142:0/898935517 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f68e404b060 con 0x7f68e80c3dc0 2024-02-21T23:24:53.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.870+0000 7f68de7fc640 1 -- 172.21.15.142:0/898935517 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f68e404dab0 con 0x7f68e80c3dc0 2024-02-21T23:24:53.904 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:53.918+0000 7f0e50021040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-21T23:24:53.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:53.969+0000 7f68de7fc640 1 -- 172.21.15.142:0/898935517 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f68e403f070 con 0x7f68e80c3dc0 2024-02-21T23:24:53.967 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:53.980+0000 7f8a1d6da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-21T23:24:54.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.040+0000 7f68effaa640 1 -- 172.21.15.142:0/898935517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a3e6bc6d-31fa-4304-ae53-07f0bfa88924", "id": 7} v 0) v1 -- 0x7f68e81478d0 con 0x7f68e80c3dc0 2024-02-21T23:24:54.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.094+0000 7f68de7fc640 1 -- 172.21.15.142:0/898935517 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a3e6bc6d-31fa-4304-ae53-07f0bfa88924", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f68e4044590 con 0x7f68e80c3dc0 2024-02-21T23:24:54.080 INFO:teuthology.orchestra.run.smithi142.stdout:7 2024-02-21T23:24:54.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.094+0000 7f68effaa640 1 -- 172.21.15.142:0/898935517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e80c3dc0 msgr2=0x7f68e80c3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:54.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.094+0000 7f68effaa640 1 --2- 172.21.15.142:0/898935517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e80c3dc0 0x7f68e80c3170 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f68e40098c0 tx=0x7f68e40046c0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.095+0000 7f68effaa640 1 -- 172.21.15.142:0/898935517 shutdown_connections 2024-02-21T23:24:54.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.095+0000 7f68effaa640 1 --2- 172.21.15.142:0/898935517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68e8150d30 0x7f68e813fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.095+0000 7f68effaa640 1 --2- 172.21.15.142:0/898935517 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68e80c41a0 0x7f68e80cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.095+0000 7f68effaa640 1 --2- 172.21.15.142:0/898935517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e80c3dc0 0x7f68e80c3170 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.095+0000 7f68effaa640 1 -- 172.21.15.142:0/898935517 >> 172.21.15.142:0/898935517 conn(0x7f68e8061b50 msgr2=0x7f68e806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:54.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.095+0000 7f68effaa640 1 -- 172.21.15.142:0/898935517 shutdown_connections 2024-02-21T23:24:54.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-21T23:24:54.095+0000 7f68effaa640 1 -- 172.21.15.142:0/898935517 wait complete. 2024-02-21T23:24:54.092 INFO:tasks.ceph.osd.0:Restarting daemon 2024-02-21T23:24:54.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 0 2024-02-21T23:24:54.095 INFO:tasks.ceph.osd.0:Started 2024-02-21T23:24:54.095 INFO:tasks.ceph.osd.1:Restarting daemon 2024-02-21T23:24:54.095 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 1 2024-02-21T23:24:54.097 INFO:tasks.ceph.osd.1:Started 2024-02-21T23:24:54.097 INFO:tasks.ceph.osd.2:Restarting daemon 2024-02-21T23:24:54.097 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 2 2024-02-21T23:24:54.099 INFO:tasks.ceph.osd.2:Started 2024-02-21T23:24:54.100 INFO:tasks.ceph.osd.3:Restarting daemon 2024-02-21T23:24:54.100 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 3 2024-02-21T23:24:54.102 INFO:tasks.ceph.osd.3:Started 2024-02-21T23:24:54.103 INFO:tasks.ceph.osd.4:Restarting daemon 2024-02-21T23:24:54.103 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 4 2024-02-21T23:24:54.105 INFO:tasks.ceph.osd.4:Started 2024-02-21T23:24:54.105 INFO:tasks.ceph.osd.5:Restarting daemon 2024-02-21T23:24:54.105 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 5 2024-02-21T23:24:54.107 INFO:tasks.ceph.osd.5:Started 2024-02-21T23:24:54.107 INFO:tasks.ceph.osd.6:Restarting daemon 2024-02-21T23:24:54.108 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 6 2024-02-21T23:24:54.109 INFO:tasks.ceph.osd.6:Started 2024-02-21T23:24:54.110 INFO:tasks.ceph.osd.7:Restarting daemon 2024-02-21T23:24:54.110 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 7 2024-02-21T23:24:54.112 INFO:tasks.ceph.osd.7:Started 2024-02-21T23:24:54.112 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:24:54.213 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:54.227+0000 7f0e50021040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-21T23:24:54.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.241+0000 7f51e7c05640 1 Processor -- start 2024-02-21T23:24:54.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.241+0000 7f51e7c05640 1 -- start start 2024-02-21T23:24:54.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.242+0000 7f51e7c05640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f51e00c6f50 0x7f51e00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:54.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.242+0000 7f51e7c05640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e015ab40 con 0x7f51e0154600 2024-02-21T23:24:54.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.242+0000 7f51e7c05640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e015ae00 con 0x7f51e00c7c10 2024-02-21T23:24:54.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.242+0000 7f51e7c05640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e00c74a0 con 0x7f51e00c6f50 2024-02-21T23:24:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.256+0000 7f51e51a0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f51e00c6f50 0x7f51e00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.256+0000 7f51e59a1640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f51e0154600 0x7f51e00ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58698/0 (socket says 172.21.15.133:58698) 2024-02-21T23:24:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.256+0000 7f51e59a1640 1 -- 172.21.15.133:0/1649269924 learned_addr learned my addr 172.21.15.133:0/1649269924 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:24:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.260+0000 7f51e51a0640 1 -- 172.21.15.133:0/1649269924 >> v1:172.21.15.142:6789/0 conn(0x7f51e00c7c10 legacy=0x7f51e00c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.260+0000 7f51e51a0640 1 -- 172.21.15.133:0/1649269924 >> v1:172.21.15.133:6789/0 conn(0x7f51e0154600 legacy=0x7f51e00ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.260+0000 7f51e51a0640 1 -- 172.21.15.133:0/1649269924 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00cb580 con 0x7f51e00c6f50 2024-02-21T23:24:54.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.265+0000 7f51e51a0640 1 --2- 172.21.15.133:0/1649269924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 0x7f51e00cbc00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f51d00097a0 tx=0x7f51d002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26e5e859645b5fec server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.271+0000 7f51e499f640 1 -- 172.21.15.133:0/1649269924 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0034070 con 0x7f51e00c6f50 2024-02-21T23:24:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.272+0000 7f51e499f640 1 -- 172.21.15.133:0/1649269924 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0030610 con 0x7f51e00c6f50 2024-02-21T23:24:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.273+0000 7f51e499f640 1 -- 172.21.15.133:0/1649269924 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00309e0 con 0x7f51e00c6f50 2024-02-21T23:24:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.273+0000 7f51e7c05640 1 -- 172.21.15.133:0/1649269924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 msgr2=0x7f51e00cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.273+0000 7f51e7c05640 1 --2- 172.21.15.133:0/1649269924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 0x7f51e00cbc00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f51d00097a0 tx=0x7f51d002b960 comp rx=0 tx=0).stop 2024-02-21T23:24:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.273+0000 7f51e7c05640 1 -- 172.21.15.133:0/1649269924 shutdown_connections 2024-02-21T23:24:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.273+0000 7f51e7c05640 1 --2- 172.21.15.133:0/1649269924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 0x7f51e00cbc00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.273+0000 7f51e7c05640 1 -- 172.21.15.133:0/1649269924 >> 172.21.15.133:0/1649269924 conn(0x7f51e0062000 msgr2=0x7f51e00cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.273+0000 7f51e7c05640 1 -- 172.21.15.133:0/1649269924 shutdown_connections 2024-02-21T23:24:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.273+0000 7f51e7c05640 1 -- 172.21.15.133:0/1649269924 wait complete. 2024-02-21T23:24:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.274+0000 7f51e7c05640 1 Processor -- start 2024-02-21T23:24:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.274+0000 7f51e7c05640 1 -- start start 2024-02-21T23:24:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.275+0000 7f51e7c05640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 0x7f51e0159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.275+0000 7f51e7c05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e00c7c10 0x7f51e00c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.275+0000 7f51e7c05640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e0154600 0x7f51e01568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.275+0000 7f51e7c05640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b5720 con 0x7f51e00c7c10 2024-02-21T23:24:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.275+0000 7f51e7c05640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00c7780 con 0x7f51e00c6f50 2024-02-21T23:24:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.275+0000 7f51e7c05640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b58a0 con 0x7f51e0154600 2024-02-21T23:24:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.275+0000 7f51e51a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e00c7c10 0x7f51e00c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.275+0000 7f51e59a1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 0x7f51e0159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.276+0000 7f51e61a2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e0154600 0x7f51e01568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.276+0000 7f51e51a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e00c7c10 0x7f51e00c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60432/0 (socket says 172.21.15.133:60432) 2024-02-21T23:24:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.276+0000 7f51e51a0640 1 -- 172.21.15.133:0/3864154855 learned_addr learned my addr 172.21.15.133:0/3864154855 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:24:54.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.279+0000 7f51e51a0640 1 -- 172.21.15.133:0/3864154855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e0154600 msgr2=0x7f51e01568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:54.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.279+0000 7f51e51a0640 1 --2- 172.21.15.133:0/3864154855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e0154600 0x7f51e01568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.279+0000 7f51e51a0640 1 -- 172.21.15.133:0/3864154855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 msgr2=0x7f51e0159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:54.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.279+0000 7f51e51a0640 1 --2- 172.21.15.133:0/3864154855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 0x7f51e0159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.279+0000 7f51e51a0640 1 -- 172.21.15.133:0/3864154855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e01f3510 con 0x7f51e00c7c10 2024-02-21T23:24:54.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.279+0000 7f51e59a1640 1 --2- 172.21.15.133:0/3864154855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 0x7f51e0159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:24:54.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.281+0000 7f51e51a0640 1 --2- 172.21.15.133:0/3864154855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e00c7c10 0x7f51e00c5b70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f51d00098c0 tx=0x7f51d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:54.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.285+0000 7f51d67fc640 1 -- 172.21.15.133:0/3864154855 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0034040 con 0x7f51e00c7c10 2024-02-21T23:24:54.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.285+0000 7f51d67fc640 1 -- 172.21.15.133:0/3864154855 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d003ace0 con 0x7f51e00c7c10 2024-02-21T23:24:54.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.285+0000 7f51e7c05640 1 -- 172.21.15.133:0/3864154855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e01f37f0 con 0x7f51e00c7c10 2024-02-21T23:24:54.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.285+0000 7f51d67fc640 1 -- 172.21.15.133:0/3864154855 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0006e10 con 0x7f51e00c7c10 2024-02-21T23:24:54.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.285+0000 7f51e7c05640 1 -- 172.21.15.133:0/3864154855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e01f3db0 con 0x7f51e00c7c10 2024-02-21T23:24:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.286+0000 7f51e7c05640 1 -- 172.21.15.133:0/3864154855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e006a130 con 0x7f51e00c7c10 2024-02-21T23:24:54.276 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:54.290+0000 7f0e50021040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-21T23:24:54.297 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:54.310+0000 7f8a1d6da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-21T23:24:54.350 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:54.363+0000 7f8a1d6da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-21T23:24:54.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.389+0000 7f51d67fc640 1 -- 172.21.15.133:0/3864154855 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f51d004b060 con 0x7f51e00c7c10 2024-02-21T23:24:54.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.389+0000 7f51d67fc640 1 -- 172.21.15.133:0/3864154855 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f51d004db20 con 0x7f51e00c7c10 2024-02-21T23:24:54.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.389+0000 7f51d67fc640 1 -- 172.21.15.133:0/3864154855 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f51d003f070 con 0x7f51e00c7c10 2024-02-21T23:24:54.390 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:54.403+0000 7f0e50021040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-21T23:24:54.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.450+0000 7f51e7c05640 1 -- 172.21.15.133:0/3864154855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51e01e5cb0 con 0x7f51e00c7c10 2024-02-21T23:24:54.473 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-02-21T23:24:54.486+0000 7f8a1d6da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-21T23:24:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.535+0000 7f51d67fc640 1 -- 172.21.15.133:0/3864154855 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f51d00305a0 con 0x7f51e00c7c10 2024-02-21T23:24:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:24:54.523 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:24:54.046821+0000","last_up_change":"0.000000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:24:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.536+0000 7f51e7c05640 1 -- 172.21.15.133:0/3864154855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e00c7c10 msgr2=0x7f51e00c5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.536+0000 7f51e7c05640 1 --2- 172.21.15.133:0/3864154855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e00c7c10 0x7f51e00c5b70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f51d00098c0 tx=0x7f51d00046c0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.536+0000 7f51e7c05640 1 -- 172.21.15.133:0/3864154855 shutdown_connections 2024-02-21T23:24:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.536+0000 7f51e7c05640 1 --2- 172.21.15.133:0/3864154855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51e0154600 0x7f51e01568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.536+0000 7f51e7c05640 1 --2- 172.21.15.133:0/3864154855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e00c7c10 0x7f51e00c5b70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.536+0000 7f51e7c05640 1 --2- 172.21.15.133:0/3864154855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51e00c6f50 0x7f51e0159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.536+0000 7f51e7c05640 1 -- 172.21.15.133:0/3864154855 >> 172.21.15.133:0/3864154855 conn(0x7f51e0062000 msgr2=0x7f51e014db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.536+0000 7f51e7c05640 1 -- 172.21.15.133:0/3864154855 shutdown_connections 2024-02-21T23:24:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:24:54.536+0000 7f51e7c05640 1 -- 172.21.15.133:0/3864154855 wait complete. 2024-02-21T23:24:54.533 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-02-21T23:24:54.534 INFO:tasks.ceph:Waiting for OSDs to come up 2024-02-21T23:24:54.705 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-02-21T23:24:54.718+0000 7f441fe68640 -1 Falling back to public interface 2024-02-21T23:24:54.726 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-02-21T23:24:54.738+0000 7fd75ad0f640 -1 Falling back to public interface 2024-02-21T23:24:54.742 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-02-21T23:24:54.754+0000 7f6018706640 -1 Falling back to public interface 2024-02-21T23:24:54.756 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-02-21T23:24:54.769+0000 7f6a264ce640 -1 Falling back to public interface 2024-02-21T23:24:54.762 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-02-21T23:24:54.775+0000 7faeceb81640 -1 Falling back to public interface 2024-02-21T23:24:54.770 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-02-21T23:24:54.783+0000 7f2966bf8640 -1 Falling back to public interface 2024-02-21T23:24:54.772 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-02-21T23:24:54.785+0000 7f542b6d9640 -1 Falling back to public interface 2024-02-21T23:24:54.778 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-02-21T23:24:54.791+0000 7ff24ab9e640 -1 Falling back to public interface 2024-02-21T23:24:55.339 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-21T23:24:55.429 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.442+0000 7fa43a38c640 1 Processor -- start 2024-02-21T23:24:55.429 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.442+0000 7fa43a38c640 1 -- start start 2024-02-21T23:24:55.429 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.443+0000 7fa43a38c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa434142a10 0x7fa434142df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:55.430 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.443+0000 7fa43a38c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43415a2d0 con 0x7fa4340ca0a0 2024-02-21T23:24:55.430 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.443+0000 7fa43a38c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43415a5b0 con 0x7fa43406f0c0 2024-02-21T23:24:55.430 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.443+0000 7fa43a38c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43415aee0 con 0x7fa434142a10 2024-02-21T23:24:55.430 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.443+0000 7fa4337fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa4340ca0a0 0x7fa43406e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58710/0 (socket says 172.21.15.133:58710) 2024-02-21T23:24:55.430 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.443+0000 7fa4337fe640 1 -- 172.21.15.133:0/4178806737 learned_addr learned my addr 172.21.15.133:0/4178806737 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:24:55.430 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.444+0000 7fa433fff640 1 --2- 172.21.15.133:0/4178806737 >> v2:172.21.15.142:3300/0 conn(0x7fa434142a10 0x7fa434142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:55.432 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.446+0000 7fa432ffd640 1 -- 172.21.15.133:0/4178806737 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389981800 0 0) 0x7fa43415a2d0 con 0x7fa4340ca0a0 2024-02-21T23:24:55.432 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.446+0000 7fa432ffd640 1 -- 172.21.15.133:0/4178806737 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41c003520 con 0x7fa4340ca0a0 2024-02-21T23:24:55.434 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.448+0000 7fa433fff640 1 -- 172.21.15.133:0/4178806737 >> v1:172.21.15.142:6789/0 conn(0x7fa43406f0c0 legacy=0x7fa43406c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:55.434 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.448+0000 7fa433fff640 1 -- 172.21.15.133:0/4178806737 >> v1:172.21.15.133:6789/0 conn(0x7fa4340ca0a0 legacy=0x7fa43406e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:55.435 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.448+0000 7fa433fff640 1 -- 172.21.15.133:0/4178806737 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43415a890 con 0x7fa434142a10 2024-02-21T23:24:55.436 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.450+0000 7fa433fff640 1 --2- 172.21.15.133:0/4178806737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa434142a10 0x7fa434142df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097a0 tx=0x7fa42402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d52a61ba4e461a8e server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:55.441 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.454+0000 7fa432ffd640 1 -- 172.21.15.133:0/4178806737 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42402f810 con 0x7fa434142a10 2024-02-21T23:24:55.441 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.455+0000 7fa432ffd640 1 -- 172.21.15.133:0/4178806737 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42402f9b0 con 0x7fa434142a10 2024-02-21T23:24:55.441 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.455+0000 7fa432ffd640 1 -- 172.21.15.133:0/4178806737 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42402fd80 con 0x7fa434142a10 2024-02-21T23:24:55.442 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.455+0000 7fa43a38c640 1 -- 172.21.15.133:0/4178806737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa434142a10 msgr2=0x7fa434142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:55.442 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.455+0000 7fa43a38c640 1 --2- 172.21.15.133:0/4178806737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa434142a10 0x7fa434142df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097a0 tx=0x7fa42402b9f0 comp rx=0 tx=0).stop 2024-02-21T23:24:55.442 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.455+0000 7fa43a38c640 1 -- 172.21.15.133:0/4178806737 shutdown_connections 2024-02-21T23:24:55.442 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.455+0000 7fa43a38c640 1 --2- 172.21.15.133:0/4178806737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa434142a10 0x7fa434142df0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:55.442 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.455+0000 7fa43a38c640 1 -- 172.21.15.133:0/4178806737 >> 172.21.15.133:0/4178806737 conn(0x7fa43406b580 msgr2=0x7fa434070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:55.442 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.455+0000 7fa43a38c640 1 -- 172.21.15.133:0/4178806737 shutdown_connections 2024-02-21T23:24:55.443 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.455+0000 7fa43a38c640 1 -- 172.21.15.133:0/4178806737 wait complete. 2024-02-21T23:24:55.443 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.456+0000 7fa43a38c640 1 Processor -- start 2024-02-21T23:24:55.443 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.457+0000 7fa43a38c640 1 -- start start 2024-02-21T23:24:55.444 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.457+0000 7fa43a38c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa43406f0c0 0x7fa4341415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:55.444 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.457+0000 7fa43a38c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4340ca0a0 0x7fa4341f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:55.444 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.457+0000 7fa43a38c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa434142a10 0x7fa4340c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:24:55.444 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.457+0000 7fa43a38c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43415a080 con 0x7fa434142a10 2024-02-21T23:24:55.444 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.457+0000 7fa43a38c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43406cd80 con 0x7fa4340ca0a0 2024-02-21T23:24:55.445 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.457+0000 7fa43a38c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4341434d0 con 0x7fa43406f0c0 2024-02-21T23:24:55.445 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.458+0000 7fa438929640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa434142a10 0x7fa4340c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:55.445 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.458+0000 7fa4337fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4340ca0a0 0x7fa4341f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:55.445 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.458+0000 7fa433fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa43406f0c0 0x7fa4341415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:24:55.445 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.458+0000 7fa438929640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa434142a10 0x7fa4340c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60434/0 (socket says 172.21.15.133:60434) 2024-02-21T23:24:55.445 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.458+0000 7fa438929640 1 -- 172.21.15.133:0/1840002439 learned_addr learned my addr 172.21.15.133:0/1840002439 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:24:55.448 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.461+0000 7fa438929640 1 -- 172.21.15.133:0/1840002439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa43406f0c0 msgr2=0x7fa4341415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:55.448 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.461+0000 7fa438929640 1 --2- 172.21.15.133:0/1840002439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa43406f0c0 0x7fa4341415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:55.448 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.461+0000 7fa438929640 1 -- 172.21.15.133:0/1840002439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4340ca0a0 msgr2=0x7fa4341f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:55.448 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.461+0000 7fa438929640 1 --2- 172.21.15.133:0/1840002439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4340ca0a0 0x7fa4341f05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:55.448 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.461+0000 7fa438929640 1 -- 172.21.15.133:0/1840002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340c4de0 con 0x7fa434142a10 2024-02-21T23:24:55.448 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.461+0000 7fa4337fe640 1 --2- 172.21.15.133:0/1840002439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4340ca0a0 0x7fa4341f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:24:55.450 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.463+0000 7fa438929640 1 --2- 172.21.15.133:0/1840002439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa434142a10 0x7fa4340c6160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00c7a0 tx=0x7fa42c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:24:55.452 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.465+0000 7fa430ff9640 1 -- 172.21.15.133:0/1840002439 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c003b70 con 0x7fa434142a10 2024-02-21T23:24:55.452 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.465+0000 7fa43a38c640 1 -- 172.21.15.133:0/1840002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4340c5140 con 0x7fa434142a10 2024-02-21T23:24:55.452 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.465+0000 7fa43a38c640 1 -- 172.21.15.133:0/1840002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4340c5700 con 0x7fa434142a10 2024-02-21T23:24:55.453 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.466+0000 7fa43a38c640 1 -- 172.21.15.133:0/1840002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43404f290 con 0x7fa434142a10 2024-02-21T23:24:55.453 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.467+0000 7fa430ff9640 1 -- 172.21.15.133:0/1840002439 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c003d10 con 0x7fa434142a10 2024-02-21T23:24:55.453 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.467+0000 7fa430ff9640 1 -- 172.21.15.133:0/1840002439 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c00aa50 con 0x7fa434142a10 2024-02-21T23:24:55.553 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.566+0000 7fa430ff9640 1 -- 172.21.15.133:0/1840002439 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 2) v1 ==== 44915+0+0 (secure 0 0 0) 0x7fa42c00abf0 con 0x7fa434142a10 2024-02-21T23:24:55.560 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.574+0000 7fa430ff9640 1 -- 172.21.15.133:0/1840002439 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fa42c01b5c0 con 0x7fa434142a10 2024-02-21T23:24:55.560 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.574+0000 7fa430ff9640 1 -- 172.21.15.133:0/1840002439 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa42c038e80 con 0x7fa434142a10 2024-02-21T23:24:55.624 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.637+0000 7fa43a38c640 1 -- 172.21.15.133:0/1840002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4341547d0 con 0x7fa434142a10 2024-02-21T23:24:55.644 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.657+0000 7fa430ff9640 1 -- 172.21.15.133:0/1840002439 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fa42c03a030 con 0x7fa434142a10 2024-02-21T23:24:55.644 INFO:teuthology.misc.health.smithi133.stdout: 2024-02-21T23:24:55.645 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":10,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:24:54.046821+0000","last_up_change":"0.000000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:24:55.645 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.658+0000 7fa43a38c640 1 -- 172.21.15.133:0/1840002439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa434142a10 msgr2=0x7fa4340c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:24:55.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.658+0000 7fa43a38c640 1 --2- 172.21.15.133:0/1840002439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa434142a10 0x7fa4340c6160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00c7a0 tx=0x7fa42c00cc60 comp rx=0 tx=0).stop 2024-02-21T23:24:55.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.659+0000 7fa43a38c640 1 -- 172.21.15.133:0/1840002439 shutdown_connections 2024-02-21T23:24:55.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.659+0000 7fa43a38c640 1 --2- 172.21.15.133:0/1840002439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa434142a10 0x7fa4340c6160 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:55.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.659+0000 7fa43a38c640 1 --2- 172.21.15.133:0/1840002439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4340ca0a0 0x7fa4341f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:55.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.659+0000 7fa43a38c640 1 --2- 172.21.15.133:0/1840002439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa43406f0c0 0x7fa4341415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:24:55.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.659+0000 7fa43a38c640 1 -- 172.21.15.133:0/1840002439 >> 172.21.15.133:0/1840002439 conn(0x7fa43406b580 msgr2=0x7fa434158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:24:55.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.659+0000 7fa43a38c640 1 -- 172.21.15.133:0/1840002439 shutdown_connections 2024-02-21T23:24:55.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:24:55.659+0000 7fa43a38c640 1 -- 172.21.15.133:0/1840002439 wait complete. 2024-02-21T23:24:55.655 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-02-21T23:24:55.928 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:24:55.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:24:56.070 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:56.083+0000 7f0df0624640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-21T23:24:56.102 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:24:56.115+0000 7f0df0624640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-02-21T23:24:58.174 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-02-21T23:24:58.187+0000 7f441fe68640 -1 osd.0 0 log_to_monitors true 2024-02-21T23:24:58.187 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-02-21T23:24:58.200+0000 7fd75ad0f640 -1 osd.6 0 log_to_monitors true 2024-02-21T23:24:58.203 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-02-21T23:24:58.216+0000 7faeceb81640 -1 osd.1 0 log_to_monitors true 2024-02-21T23:24:58.218 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-02-21T23:24:58.232+0000 7f6018706640 -1 osd.7 0 log_to_monitors true 2024-02-21T23:24:58.236 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-02-21T23:24:58.249+0000 7f2966bf8640 -1 osd.2 0 log_to_monitors true 2024-02-21T23:24:58.266 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-02-21T23:24:58.279+0000 7f542b6d9640 -1 osd.4 0 log_to_monitors true 2024-02-21T23:24:58.266 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-02-21T23:24:58.279+0000 7f6a264ce640 -1 osd.5 0 log_to_monitors true 2024-02-21T23:24:58.274 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-02-21T23:24:58.288+0000 7ff24ab9e640 -1 osd.3 0 log_to_monitors true 2024-02-21T23:25:01.338 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-02-21T23:25:01.353+0000 7ff2467f7640 -1 osd.3 0 waiting for initial osdmap 2024-02-21T23:25:01.350 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-02-21T23:25:01.364+0000 7f29627f7640 -1 osd.2 0 waiting for initial osdmap 2024-02-21T23:25:01.356 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-02-21T23:25:01.370+0000 7fd7571f8640 -1 osd.6 0 waiting for initial osdmap 2024-02-21T23:25:01.361 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-02-21T23:25:01.375+0000 7f6a221f7640 -1 osd.5 0 waiting for initial osdmap 2024-02-21T23:25:01.374 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-02-21T23:25:01.388+0000 7f60143f7640 -1 osd.7 0 waiting for initial osdmap 2024-02-21T23:25:01.380 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-02-21T23:25:01.393+0000 7faecaff8640 -1 osd.1 0 waiting for initial osdmap 2024-02-21T23:25:01.380 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-02-21T23:25:01.394+0000 7f5427bf8640 -1 osd.4 0 waiting for initial osdmap 2024-02-21T23:25:01.404 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-02-21T23:25:01.418+0000 7f6a1a080640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-21T23:25:01.405 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-02-21T23:25:01.420+0000 7f295a680640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-21T23:25:01.417 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-02-21T23:25:01.432+0000 7ff23e680640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-21T23:25:01.419 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-02-21T23:25:01.434+0000 7f441bbf7640 -1 osd.0 0 waiting for initial osdmap 2024-02-21T23:25:01.431 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-02-21T23:25:01.445+0000 7fd74e880640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-21T23:25:01.434 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-02-21T23:25:01.449+0000 7faec2680640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-21T23:25:01.446 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-02-21T23:25:01.460+0000 7f541f280640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-21T23:25:01.446 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-02-21T23:25:01.460+0000 7f600c280640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-21T23:25:01.453 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-02-21T23:25:01.468+0000 7f4413a17640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-21T23:25:02.462 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:02.553 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.567+0000 7f6929a20640 1 Processor -- start 2024-02-21T23:25:02.553 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.567+0000 7f6929a20640 1 -- start start 2024-02-21T23:25:02.554 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.568+0000 7f6929a20640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f69240c6ae0 0x7f69240c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:02.554 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.568+0000 7f6929a20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f692406f690 0x7f69240c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:02.554 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.568+0000 7f6929a20640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69240cb300 con 0x7f6924150910 2024-02-21T23:25:02.554 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.568+0000 7f6929a20640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692406fae0 con 0x7f69240c6ae0 2024-02-21T23:25:02.554 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.568+0000 7f6929a20640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69240c4590 con 0x7f692406f690 2024-02-21T23:25:02.555 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.569+0000 7f691bfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f69240c6ae0 0x7f69240c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:02.555 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.569+0000 7f6922ffd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f6924150910 0x7f69240c5ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57014/0 (socket says 172.21.15.133:57014) 2024-02-21T23:25:02.555 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.569+0000 7f6922ffd640 1 -- 172.21.15.133:0/673046614 learned_addr learned my addr 172.21.15.133:0/673046614 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:02.559 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.572+0000 7f691bfff640 1 -- 172.21.15.133:0/673046614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f692406f690 msgr2=0x7f69240c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:02.559 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.572+0000 7f691bfff640 1 --2- 172.21.15.133:0/673046614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f692406f690 0x7f69240c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:02.559 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.572+0000 7f691bfff640 1 -- 172.21.15.133:0/673046614 >> v1:172.21.15.142:6789/0 conn(0x7f6924150910 legacy=0x7f69240c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:02.559 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.572+0000 7f691bfff640 1 -- 172.21.15.133:0/673046614 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69240c5290 con 0x7f69240c6ae0 2024-02-21T23:25:02.561 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.575+0000 7f691bfff640 1 --2- 172.21.15.133:0/673046614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69240c6ae0 0x7f69240c4a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f690c0097a0 tx=0x7f690c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b5e50b16c8ee801 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:02.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.658+0000 7f69227fc640 1 -- 172.21.15.133:0/673046614 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c034070 con 0x7f69240c6ae0 2024-02-21T23:25:02.646 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.658+0000 7f69227fc640 1 -- 172.21.15.133:0/673046614 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c030610 con 0x7f69240c6ae0 2024-02-21T23:25:02.647 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.658+0000 7f69227fc640 1 -- 172.21.15.133:0/673046614 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c0309e0 con 0x7f69240c6ae0 2024-02-21T23:25:02.647 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.659+0000 7f6929a20640 1 -- 172.21.15.133:0/673046614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69240c6ae0 msgr2=0x7f69240c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:02.647 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.659+0000 7f6929a20640 1 --2- 172.21.15.133:0/673046614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69240c6ae0 0x7f69240c4a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f690c0097a0 tx=0x7f690c02b960 comp rx=0 tx=0).stop 2024-02-21T23:25:02.647 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.659+0000 7f6929a20640 1 -- 172.21.15.133:0/673046614 shutdown_connections 2024-02-21T23:25:02.647 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.659+0000 7f6929a20640 1 --2- 172.21.15.133:0/673046614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f692406f690 0x7f69240c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:02.647 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.659+0000 7f6929a20640 1 --2- 172.21.15.133:0/673046614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69240c6ae0 0x7f69240c4a20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:02.648 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.659+0000 7f6929a20640 1 -- 172.21.15.133:0/673046614 >> 172.21.15.133:0/673046614 conn(0x7f69240cb540 msgr2=0x7f69240cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:02.648 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.659+0000 7f6929a20640 1 -- 172.21.15.133:0/673046614 shutdown_connections 2024-02-21T23:25:02.648 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.659+0000 7f6929a20640 1 -- 172.21.15.133:0/673046614 wait complete. 2024-02-21T23:25:02.648 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.660+0000 7f6929a20640 1 Processor -- start 2024-02-21T23:25:02.648 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.660+0000 7f6929a20640 1 -- start start 2024-02-21T23:25:02.650 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.661+0000 7f6929a20640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692406f690 0x7f69241e0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:02.650 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.661+0000 7f6929a20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69240c6ae0 0x7f69241dc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:02.650 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.661+0000 7f6929a20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6924150910 0x7f69241dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:02.651 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.661+0000 7f6929a20640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6924150cf0 con 0x7f6924150910 2024-02-21T23:25:02.651 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.661+0000 7f6929a20640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69240c5850 con 0x7f692406f690 2024-02-21T23:25:02.651 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.661+0000 7f6929a20640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69240c4200 con 0x7f69240c6ae0 2024-02-21T23:25:02.651 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.661+0000 7f691bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69240c6ae0 0x7f69241dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:02.651 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.662+0000 7f691bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69240c6ae0 0x7f69241dc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:44104/0 (socket says 172.21.15.133:44104) 2024-02-21T23:25:02.651 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.662+0000 7f691bfff640 1 -- 172.21.15.133:0/3278259278 learned_addr learned my addr 172.21.15.133:0/3278259278 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:02.652 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.662+0000 7f6922ffd640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692406f690 0x7f69241e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:02.653 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.667+0000 7f691bfff640 1 -- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692406f690 msgr2=0x7f69241e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:02.653 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.667+0000 7f691bfff640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692406f690 0x7f69241e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:02.653 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.667+0000 7f691bfff640 1 -- 172.21.15.133:0/3278259278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6924150910 msgr2=0x7f69241dcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:02.653 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.667+0000 7f691bfff640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6924150910 0x7f69241dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:02.653 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.667+0000 7f691bfff640 1 -- 172.21.15.133:0/3278259278 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69240718f0 con 0x7f69240c6ae0 2024-02-21T23:25:02.656 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.671+0000 7f691bfff640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69240c6ae0 0x7f69241dc660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f690c02bf70 tx=0x7f690c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:02.661 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.674+0000 7f6920ff9640 1 -- 172.21.15.133:0/3278259278 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c034040 con 0x7f69240c6ae0 2024-02-21T23:25:02.661 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.675+0000 7f6920ff9640 1 -- 172.21.15.133:0/3278259278 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c039a10 con 0x7f69240c6ae0 2024-02-21T23:25:02.661 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.675+0000 7f6920ff9640 1 -- 172.21.15.133:0/3278259278 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c039de0 con 0x7f69240c6ae0 2024-02-21T23:25:02.661 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.675+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692413fb00 con 0x7f69240c6ae0 2024-02-21T23:25:02.661 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.675+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6924140080 con 0x7f69240c6ae0 2024-02-21T23:25:02.662 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.675+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692404f290 con 0x7f69240c6ae0 2024-02-21T23:25:02.680 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.694+0000 7f6920ff9640 1 -- 172.21.15.133:0/3278259278 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f690c037070 con 0x7f69240c6ae0 2024-02-21T23:25:02.680 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.694+0000 7f6920ff9640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f690406b490 0x7f690406d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:02.680 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.694+0000 7f6922ffd640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f690406b490 0x7f690406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:02.680 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.695+0000 7f6922ffd640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f690406b490 0x7f690406d950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6914007fd0 tx=0x7f6914007e80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:02.682 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:02.696+0000 7f6920ff9640 1 -- 172.21.15.133:0/3278259278 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f690c0305a0 con 0x7f69240c6ae0 2024-02-21T23:25:03.033 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.047+0000 7f6920ff9640 1 -- 172.21.15.133:0/3278259278 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f690c07bb60 con 0x7f69240c6ae0 2024-02-21T23:25:03.212 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.226+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69240caa30 con 0x7f69240c6ae0 2024-02-21T23:25:03.265 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.279+0000 7f6920ff9640 1 -- 172.21.15.133:0/3278259278 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10957 (secure 0 0 0) 0x7f690c07b460 con 0x7f69240c6ae0 2024-02-21T23:25:03.265 INFO:teuthology.misc.health.smithi133.stdout: 2024-02-21T23:25:03.266 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":13,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:02.449240+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f690406b490 msgr2=0x7f690406d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f690406b490 0x7f690406d950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6914007fd0 tx=0x7f6914007e80 comp rx=0 tx=0).stop 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69240c6ae0 msgr2=0x7f69241dc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69240c6ae0 0x7f69241dc660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f690c02bf70 tx=0x7f690c0046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 shutdown_connections 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f690406b490 0x7f690406d950 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6924150910 0x7f69241dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69240c6ae0 0x7f69241dc660 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 --2- 172.21.15.133:0/3278259278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692406f690 0x7f69241e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 >> 172.21.15.133:0/3278259278 conn(0x7f69240cb540 msgr2=0x7f69240cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 shutdown_connections 2024-02-21T23:25:03.268 INFO:teuthology.misc.health.smithi133.stderr:2024-02-21T23:25:03.282+0000 7f6929a20640 1 -- 172.21.15.133:0/3278259278 wait complete. 2024-02-21T23:25:03.278 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-02-21T23:25:03.278 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-02-21T23:25:03.278 DEBUG:tasks.ceph_manager:running mds.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-02-21T23:25:03.278 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-21T23:25:03.278 DEBUG:teuthology.orchestra.run.smithi133:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i a 2024-02-21T23:25:03.281 INFO:tasks.ceph.mds.a:Started 2024-02-21T23:25:03.281 DEBUG:tasks.ceph_manager:running mds.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-02-21T23:25:03.281 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-21T23:25:03.281 DEBUG:teuthology.orchestra.run.smithi133:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i c 2024-02-21T23:25:03.283 INFO:tasks.ceph.mds.c:Started 2024-02-21T23:25:03.284 DEBUG:tasks.ceph_manager:running mds.e under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.e.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-02-21T23:25:03.284 INFO:tasks.ceph.mds.e:Restarting daemon 2024-02-21T23:25:03.284 DEBUG:teuthology.orchestra.run.smithi133:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.e.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i e 2024-02-21T23:25:03.286 INFO:tasks.ceph.mds.e:Started 2024-02-21T23:25:03.286 DEBUG:tasks.ceph_manager:running mds.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-02-21T23:25:03.286 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-21T23:25:03.286 DEBUG:teuthology.orchestra.run.smithi142:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i b 2024-02-21T23:25:03.291 INFO:tasks.ceph.mds.b:Started 2024-02-21T23:25:03.291 DEBUG:tasks.ceph_manager:running mds.d under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.d.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-02-21T23:25:03.291 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-21T23:25:03.292 DEBUG:teuthology.orchestra.run.smithi142:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.d.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i d 2024-02-21T23:25:03.294 INFO:tasks.ceph.mds.d:Started 2024-02-21T23:25:03.294 DEBUG:tasks.ceph_manager:running mds.f under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.f.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-02-21T23:25:03.294 INFO:tasks.ceph.mds.f:Restarting daemon 2024-02-21T23:25:03.294 DEBUG:teuthology.orchestra.run.smithi142:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.f.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i f 2024-02-21T23:25:03.296 INFO:tasks.ceph.mds.f:Started 2024-02-21T23:25:03.296 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-02-21T23:25:03.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:03.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.424+0000 7f5475fdc640 1 Processor -- start 2024-02-21T23:25:03.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.424+0000 7f5475fdc640 1 -- start start 2024-02-21T23:25:03.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.426+0000 7f5475fdc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f54700703c0 0x7f54700c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:03.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.426+0000 7f5475fdc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5470070800 0x7f54700c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:03.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.426+0000 7f5475fdc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700cd570 con 0x7f5470154240 2024-02-21T23:25:03.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.426+0000 7f5475fdc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547015a130 con 0x7f54700703c0 2024-02-21T23:25:03.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.426+0000 7f5475fdc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547015a410 con 0x7f5470070800 2024-02-21T23:25:03.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.426+0000 7f546ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5470070800 0x7f54700c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:03.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.426+0000 7f546effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f54700703c0 0x7f54700c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:03.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.427+0000 7f546f7fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f5470154240 0x7f54700ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57020/0 (socket says 172.21.15.133:57020) 2024-02-21T23:25:03.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.427+0000 7f546f7fe640 1 -- 172.21.15.133:0/1083775841 learned_addr learned my addr 172.21.15.133:0/1083775841 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:03.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.429+0000 7f546effd640 1 -- 172.21.15.133:0/1083775841 >> v2:172.21.15.142:3300/0 conn(0x7f5470070800 msgr2=0x7f54700c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:03.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.429+0000 7f546effd640 1 --2- 172.21.15.133:0/1083775841 >> v2:172.21.15.142:3300/0 conn(0x7f5470070800 0x7f54700c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.429+0000 7f546effd640 1 -- 172.21.15.133:0/1083775841 >> v1:172.21.15.142:6789/0 conn(0x7f5470154240 legacy=0x7f54700ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:03.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.429+0000 7f546effd640 1 -- 172.21.15.133:0/1083775841 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547015a6f0 con 0x7f54700703c0 2024-02-21T23:25:03.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.431+0000 7f546effd640 1 --2- 172.21.15.133:0/1083775841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54700703c0 0x7f54700c90b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f547015e700 tx=0x7f546402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4a0e9972e32051 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:03.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.436+0000 7f546e7fc640 1 -- 172.21.15.133:0/1083775841 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464035020 con 0x7f54700703c0 2024-02-21T23:25:03.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.436+0000 7f546e7fc640 1 -- 172.21.15.133:0/1083775841 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5464031830 con 0x7f54700703c0 2024-02-21T23:25:03.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.436+0000 7f546e7fc640 1 -- 172.21.15.133:0/1083775841 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464031c00 con 0x7f54700703c0 2024-02-21T23:25:03.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.437+0000 7f5475fdc640 1 -- 172.21.15.133:0/1083775841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54700703c0 msgr2=0x7f54700c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:03.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.437+0000 7f5475fdc640 1 --2- 172.21.15.133:0/1083775841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54700703c0 0x7f54700c90b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f547015e700 tx=0x7f546402b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.437+0000 7f5475fdc640 1 -- 172.21.15.133:0/1083775841 shutdown_connections 2024-02-21T23:25:03.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.437+0000 7f5475fdc640 1 --2- 172.21.15.133:0/1083775841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54700703c0 0x7f54700c90b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.437+0000 7f5475fdc640 1 --2- 172.21.15.133:0/1083775841 >> v2:172.21.15.142:3300/0 conn(0x7f5470070800 0x7f54700c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.437+0000 7f5475fdc640 1 -- 172.21.15.133:0/1083775841 >> 172.21.15.133:0/1083775841 conn(0x7f5470061b50 msgr2=0x7f5470072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:03.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.437+0000 7f5475fdc640 1 -- 172.21.15.133:0/1083775841 shutdown_connections 2024-02-21T23:25:03.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.437+0000 7f5475fdc640 1 -- 172.21.15.133:0/1083775841 wait complete. 2024-02-21T23:25:03.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.439+0000 7f5475fdc640 1 Processor -- start 2024-02-21T23:25:03.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.439+0000 7f5475fdc640 1 -- start start 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.440+0000 7f5475fdc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54700703c0 0x7f54700c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.440+0000 7f5475fdc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5470070800 0x7f547014a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.440+0000 7f5475fdc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470154240 0x7f547015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.440+0000 7f5475fdc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5470154620 con 0x7f5470154240 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.440+0000 7f5475fdc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547015d610 con 0x7f5470070800 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.440+0000 7f5475fdc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54701e2ba0 con 0x7f54700703c0 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.441+0000 7f546effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5470070800 0x7f547014a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.441+0000 7f546ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470154240 0x7f547015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.441+0000 7f546effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5470070800 0x7f547014a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:47920/0 (socket says 172.21.15.133:47920) 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.441+0000 7f546effd640 1 -- 172.21.15.133:0/1654405897 learned_addr learned my addr 172.21.15.133:0/1654405897 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:03.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.441+0000 7f546f7fe640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54700703c0 0x7f54700c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:03.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.445+0000 7f546ffff640 1 -- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54700703c0 msgr2=0x7f54700c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:03.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.445+0000 7f546ffff640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54700703c0 0x7f54700c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.445+0000 7f546ffff640 1 -- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5470070800 msgr2=0x7f547014a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:03.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.445+0000 7f546ffff640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5470070800 0x7f547014a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:03.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.445+0000 7f546ffff640 1 -- 172.21.15.133:0/1654405897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54701f3dd0 con 0x7f5470154240 2024-02-21T23:25:03.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.447+0000 7f546ffff640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470154240 0x7f547015b1e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f546800efc0 tx=0x7f546800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:03.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.453+0000 7f5457fff640 1 -- 172.21.15.133:0/1654405897 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468018020 con 0x7f5470154240 2024-02-21T23:25:03.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.453+0000 7f5457fff640 1 -- 172.21.15.133:0/1654405897 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5468014480 con 0x7f5470154240 2024-02-21T23:25:03.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.453+0000 7f5457fff640 1 -- 172.21.15.133:0/1654405897 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468014870 con 0x7f5470154240 2024-02-21T23:25:03.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.453+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54700ca510 con 0x7f5470154240 2024-02-21T23:25:03.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.453+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54700caad0 con 0x7f5470154240 2024-02-21T23:25:03.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.455+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547006a130 con 0x7f5470154240 2024-02-21T23:25:03.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.486+0000 7f5457fff640 1 -- 172.21.15.133:0/1654405897 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f5468005d10 con 0x7f5470154240 2024-02-21T23:25:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.487+0000 7f5457fff640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f545c06b490 0x7f545c06d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.487+0000 7f546f7fe640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f545c06b490 0x7f545c06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.488+0000 7f546f7fe640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f545c06b490 0x7f545c06d950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5460007fd0 tx=0x7f5460007e80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:03.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.736+0000 7f5457fff640 1 -- 172.21.15.133:0/1654405897 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f5468057d30 con 0x7f5470154240 2024-02-21T23:25:03.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.736+0000 7f5457fff640 1 -- 172.21.15.133:0/1654405897 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f5468022030 con 0x7f5470154240 2024-02-21T23:25:03.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:03.971+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54700cd240 con 0x7f5470154240 2024-02-21T23:25:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.246+0000 7f5457fff640 1 -- 172.21.15.133:0/1654405897 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+12936 (secure 0 0 0) 0x7f5468010040 con 0x7f5470154240 2024-02-21T23:25:04.232 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:04.232 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":14,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:03.736732+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f545c06b490 msgr2=0x7f545c06d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f545c06b490 0x7f545c06d950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5460007fd0 tx=0x7f5460007e80 comp rx=0 tx=0).stop 2024-02-21T23:25:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470154240 msgr2=0x7f547015b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470154240 0x7f547015b1e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f546800efc0 tx=0x7f546800c490 comp rx=0 tx=0).stop 2024-02-21T23:25:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 shutdown_connections 2024-02-21T23:25:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f545c06b490 0x7f545c06d950 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470154240 0x7f547015b1e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5470070800 0x7f547014a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 --2- 172.21.15.133:0/1654405897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54700703c0 0x7f54700c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 >> 172.21.15.133:0/1654405897 conn(0x7f5470061b50 msgr2=0x7f54701472e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 shutdown_connections 2024-02-21T23:25:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.249+0000 7f5475fdc640 1 -- 172.21.15.133:0/1654405897 wait complete. 2024-02-21T23:25:04.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-21T23:25:02.902698+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-21T23:25:04.248 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-21T23:25:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.373+0000 7fad9e9e9640 1 Processor -- start 2024-02-21T23:25:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.373+0000 7fad9e9e9640 1 -- start start 2024-02-21T23:25:04.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.374+0000 7fad9e9e9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad98071950 0x7fad98072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:04.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.374+0000 7fad9e9e9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fad981584b0 0x7fad98158900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:04.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.374+0000 7fad9e9e9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad980c7370 con 0x7fad98072230 2024-02-21T23:25:04.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.374+0000 7fad9e9e9640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad98158e40 con 0x7fad98071950 2024-02-21T23:25:04.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.374+0000 7fad9e9e9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad98152640 con 0x7fad981584b0 2024-02-21T23:25:04.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.376+0000 7fad977fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad98071950 0x7fad98072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:04.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.377+0000 7fad977fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad98071950 0x7fad98072610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33884/0 (socket says 172.21.15.133:33884) 2024-02-21T23:25:04.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.377+0000 7fad977fe640 1 -- 172.21.15.133:0/748472210 learned_addr learned my addr 172.21.15.133:0/748472210 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:04.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.380+0000 7fad977fe640 1 -- 172.21.15.133:0/748472210 >> v2:172.21.15.142:3300/0 conn(0x7fad981584b0 msgr2=0x7fad98158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:04.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.380+0000 7fad977fe640 1 --2- 172.21.15.133:0/748472210 >> v2:172.21.15.142:3300/0 conn(0x7fad981584b0 0x7fad98158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:04.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.380+0000 7fad977fe640 1 -- 172.21.15.133:0/748472210 >> v1:172.21.15.142:6789/0 conn(0x7fad98072230 legacy=0x7fad980c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:04.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.380+0000 7fad977fe640 1 -- 172.21.15.133:0/748472210 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad980c7eb0 con 0x7fad98071950 2024-02-21T23:25:04.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.386+0000 7fad977fe640 1 --2- 172.21.15.133:0/748472210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98071950 0x7fad98072610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fad98073730 tx=0x7fad8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a420d3403da1bad server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:04.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.391+0000 7fad96ffd640 1 -- 172.21.15.133:0/748472210 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c035020 con 0x7fad98071950 2024-02-21T23:25:04.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.391+0000 7fad96ffd640 1 -- 172.21.15.133:0/748472210 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c031870 con 0x7fad98071950 2024-02-21T23:25:04.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.391+0000 7fad96ffd640 1 -- 172.21.15.133:0/748472210 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c031c40 con 0x7fad98071950 2024-02-21T23:25:04.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.392+0000 7fad9e9e9640 1 -- 172.21.15.133:0/748472210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98071950 msgr2=0x7fad98072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:04.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.392+0000 7fad9e9e9640 1 --2- 172.21.15.133:0/748472210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98071950 0x7fad98072610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fad98073730 tx=0x7fad8c02b910 comp rx=0 tx=0).stop 2024-02-21T23:25:04.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.392+0000 7fad9e9e9640 1 -- 172.21.15.133:0/748472210 shutdown_connections 2024-02-21T23:25:04.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.392+0000 7fad9e9e9640 1 --2- 172.21.15.133:0/748472210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98071950 0x7fad98072610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:04.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.392+0000 7fad9e9e9640 1 --2- 172.21.15.133:0/748472210 >> v2:172.21.15.142:3300/0 conn(0x7fad981584b0 0x7fad98158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:04.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.392+0000 7fad9e9e9640 1 -- 172.21.15.133:0/748472210 >> 172.21.15.133:0/748472210 conn(0x7fad9806f6e0 msgr2=0x7fad9806faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:04.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.392+0000 7fad9e9e9640 1 -- 172.21.15.133:0/748472210 shutdown_connections 2024-02-21T23:25:04.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.392+0000 7fad9e9e9640 1 -- 172.21.15.133:0/748472210 wait complete. 2024-02-21T23:25:04.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.393+0000 7fad9e9e9640 1 Processor -- start 2024-02-21T23:25:04.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.393+0000 7fad9e9e9640 1 -- start start 2024-02-21T23:25:04.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.394+0000 7fad9e9e9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad98071950 0x7fad981eeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:04.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.394+0000 7fad9e9e9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98072230 0x7fad98152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:04.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.394+0000 7fad9e9e9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad981584b0 0x7fad981f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:04.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.394+0000 7fad9e9e9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad980c78f0 con 0x7fad98072230 2024-02-21T23:25:04.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.394+0000 7fad9e9e9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad98158020 con 0x7fad981584b0 2024-02-21T23:25:04.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.394+0000 7fad9e9e9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad980c7bd0 con 0x7fad98071950 2024-02-21T23:25:04.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.394+0000 7fad977fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98072230 0x7fad98152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:04.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.395+0000 7fad977fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98072230 0x7fad98152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33886/0 (socket says 172.21.15.133:33886) 2024-02-21T23:25:04.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.395+0000 7fad977fe640 1 -- 172.21.15.133:0/1889501997 learned_addr learned my addr 172.21.15.133:0/1889501997 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:04.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.398+0000 7fad97fff640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad98071950 0x7fad981eeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:04.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.399+0000 7fad977fe640 1 -- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad98071950 msgr2=0x7fad981eeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:04.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.399+0000 7fad977fe640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad98071950 0x7fad981eeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:04.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.399+0000 7fad977fe640 1 -- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad981584b0 msgr2=0x7fad981f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:04.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.399+0000 7fad977fe640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad981584b0 0x7fad981f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:04.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.399+0000 7fad977fe640 1 -- 172.21.15.133:0/1889501997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad981f23b0 con 0x7fad98072230 2024-02-21T23:25:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.404+0000 7fad977fe640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98072230 0x7fad98152cb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad8c000c00 tx=0x7fad8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.410+0000 7fad94ff9640 1 -- 172.21.15.133:0/1889501997 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c040070 con 0x7fad98072230 2024-02-21T23:25:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.411+0000 7fad94ff9640 1 -- 172.21.15.133:0/1889501997 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c03a420 con 0x7fad98072230 2024-02-21T23:25:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.411+0000 7fad94ff9640 1 -- 172.21.15.133:0/1889501997 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c035040 con 0x7fad98072230 2024-02-21T23:25:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.411+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad980cbb40 con 0x7fad98072230 2024-02-21T23:25:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.411+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad980cb860 con 0x7fad98072230 2024-02-21T23:25:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.412+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9806a130 con 0x7fad98072230 2024-02-21T23:25:04.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.441+0000 7fad94ff9640 1 -- 172.21.15.133:0/1889501997 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fad8c038020 con 0x7fad98072230 2024-02-21T23:25:04.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.441+0000 7fad94ff9640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fad8406b470 0x7fad8406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:04.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.441+0000 7fad97fff640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fad8406b470 0x7fad8406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:04.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.442+0000 7fad97fff640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fad8406b470 0x7fad8406d930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fad880015d0 tx=0x7fad88009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:04.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.447+0000 7fad94ff9640 1 -- 172.21.15.133:0/1889501997 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4846+0+0 (secure 0 0 0) 0x7fad8c081520 con 0x7fad98072230 2024-02-21T23:25:04.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.749+0000 7fad94ff9640 1 -- 172.21.15.133:0/1889501997 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fad8c07b860 con 0x7fad98072230 2024-02-21T23:25:04.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:04.997+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad980c6020 con 0x7fad98072230 2024-02-21T23:25:05.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.020+0000 7fad94ff9640 1 -- 172.21.15.133:0/1889501997 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v14) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad8c07b160 con 0x7fad98072230 2024-02-21T23:25:05.005 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fad8406b470 msgr2=0x7fad8406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fad8406b470 0x7fad8406d930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fad880015d0 tx=0x7fad88009040 comp rx=0 tx=0).stop 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98072230 msgr2=0x7fad98152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98072230 0x7fad98152cb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad8c000c00 tx=0x7fad8c0046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 shutdown_connections 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fad8406b470 0x7fad8406d930 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad981584b0 0x7fad981f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad98072230 0x7fad98152cb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 --2- 172.21.15.133:0/1889501997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad98071950 0x7fad981eeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.024+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 >> 172.21.15.133:0/1889501997 conn(0x7fad9806f6e0 msgr2=0x7fad9814dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.025+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 shutdown_connections 2024-02-21T23:25:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.025+0000 7fad9e9e9640 1 -- 172.21.15.133:0/1889501997 wait complete. 2024-02-21T23:25:05.028 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:25:05.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.152+0000 7f951da36640 1 Processor -- start 2024-02-21T23:25:05.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.152+0000 7f951da36640 1 -- start start 2024-02-21T23:25:05.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.153+0000 7f951da36640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9518154240 0x7f95180ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:05.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.153+0000 7f951da36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 0x7f95180c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:05.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.153+0000 7f951da36640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95180cd300 con 0x7f9518070800 2024-02-21T23:25:05.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.153+0000 7f951da36640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95180cd570 con 0x7f9518154240 2024-02-21T23:25:05.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.153+0000 7f951da36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951815a130 con 0x7f95180703c0 2024-02-21T23:25:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.154+0000 7f9516ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9518154240 0x7f95180ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.154+0000 7f95177fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9518070800 0x7f951815d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35230/0 (socket says 172.21.15.133:35230) 2024-02-21T23:25:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.154+0000 7f95177fe640 1 -- 172.21.15.133:0/2353884960 learned_addr learned my addr 172.21.15.133:0/2353884960 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.154+0000 7f95167fc640 1 --2- 172.21.15.133:0/2353884960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 0x7f95180c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:05.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.156+0000 7f9516ffd640 1 -- 172.21.15.133:0/2353884960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 msgr2=0x7f95180c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:05.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.156+0000 7f9516ffd640 1 --2- 172.21.15.133:0/2353884960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 0x7f95180c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.156+0000 7f9516ffd640 1 -- 172.21.15.133:0/2353884960 >> v1:172.21.15.133:6789/0 conn(0x7f9518070800 legacy=0x7f951815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:05.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.156+0000 7f9516ffd640 1 -- 172.21.15.133:0/2353884960 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951815a410 con 0x7f9518154240 2024-02-21T23:25:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.158+0000 7f9516ffd640 1 --2- 172.21.15.133:0/2353884960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 0x7f95180ccdc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f95080097a0 tx=0x7f950802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6cbb4de9473f31a server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:05.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.161+0000 7f9515ffb640 1 -- 172.21.15.133:0/2353884960 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508035020 con 0x7f9518154240 2024-02-21T23:25:05.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.164+0000 7f9515ffb640 1 -- 172.21.15.133:0/2353884960 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9508031830 con 0x7f9518154240 2024-02-21T23:25:05.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.164+0000 7f9515ffb640 1 -- 172.21.15.133:0/2353884960 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508031c00 con 0x7f9518154240 2024-02-21T23:25:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.164+0000 7f951da36640 1 -- 172.21.15.133:0/2353884960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 msgr2=0x7f95180ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.164+0000 7f951da36640 1 --2- 172.21.15.133:0/2353884960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 0x7f95180ccdc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f95080097a0 tx=0x7f950802b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.165+0000 7f951da36640 1 -- 172.21.15.133:0/2353884960 shutdown_connections 2024-02-21T23:25:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.165+0000 7f951da36640 1 --2- 172.21.15.133:0/2353884960 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 0x7f95180c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.165+0000 7f951da36640 1 --2- 172.21.15.133:0/2353884960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 0x7f95180ccdc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.165+0000 7f951da36640 1 -- 172.21.15.133:0/2353884960 >> 172.21.15.133:0/2353884960 conn(0x7f9518061b50 msgr2=0x7f9518072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:05.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.165+0000 7f951da36640 1 -- 172.21.15.133:0/2353884960 shutdown_connections 2024-02-21T23:25:05.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.165+0000 7f951da36640 1 -- 172.21.15.133:0/2353884960 wait complete. 2024-02-21T23:25:05.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.166+0000 7f951da36640 1 Processor -- start 2024-02-21T23:25:05.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.166+0000 7f951da36640 1 -- start start 2024-02-21T23:25:05.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.167+0000 7f951da36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 0x7f951814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:05.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.167+0000 7f951da36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9518070800 0x7f951815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:05.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.167+0000 7f951da36640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 0x7f95180ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:05.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.167+0000 7f951da36640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9518154620 con 0x7f9518070800 2024-02-21T23:25:05.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.167+0000 7f951da36640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951815a9d0 con 0x7f9518154240 2024-02-21T23:25:05.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.167+0000 7f951da36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951815c460 con 0x7f95180703c0 2024-02-21T23:25:05.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.168+0000 7f95177fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 0x7f95180ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:05.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.168+0000 7f9516ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 0x7f951814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:05.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.168+0000 7f95167fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9518070800 0x7f951815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:05.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.168+0000 7f95177fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 0x7f95180ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:47954/0 (socket says 172.21.15.133:47954) 2024-02-21T23:25:05.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.168+0000 7f95177fe640 1 -- 172.21.15.133:0/2789977170 learned_addr learned my addr 172.21.15.133:0/2789977170 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:05.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.171+0000 7f95177fe640 1 -- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 msgr2=0x7f951814d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:05.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.171+0000 7f95177fe640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 0x7f951814d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.172+0000 7f95177fe640 1 -- 172.21.15.133:0/2789977170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9518070800 msgr2=0x7f951815ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:05.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.172+0000 7f95177fe640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9518070800 0x7f951815ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.172+0000 7f95177fe640 1 -- 172.21.15.133:0/2789977170 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95181f01e0 con 0x7f9518154240 2024-02-21T23:25:05.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.174+0000 7f95177fe640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 0x7f95180ca650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f951000ca80 tx=0x7f951000cf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:05.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.179+0000 7f94fffff640 1 -- 172.21.15.133:0/2789977170 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510017070 con 0x7f9518154240 2024-02-21T23:25:05.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.179+0000 7f94fffff640 1 -- 172.21.15.133:0/2789977170 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951000ac30 con 0x7f9518154240 2024-02-21T23:25:05.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.179+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95181f04c0 con 0x7f9518154240 2024-02-21T23:25:05.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.179+0000 7f94fffff640 1 -- 172.21.15.133:0/2789977170 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510012600 con 0x7f9518154240 2024-02-21T23:25:05.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.179+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95181f0a80 con 0x7f9518154240 2024-02-21T23:25:05.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.180+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951806a130 con 0x7f9518154240 2024-02-21T23:25:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.458+0000 7f94fffff640 1 -- 172.21.15.133:0/2789977170 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f951001a070 con 0x7f9518154240 2024-02-21T23:25:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.458+0000 7f94fffff640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f950406b490 0x7f950406d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.459+0000 7f9516ffd640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f950406b490 0x7f950406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.459+0000 7f9516ffd640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f950406b490 0x7f950406d950 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9508036000 tx=0x7f9508008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.464+0000 7f94fffff640 1 -- 172.21.15.133:0/2789977170 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4846+0+0 (secure 0 0 0) 0x7f9510056c00 con 0x7f9518154240 2024-02-21T23:25:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.483+0000 7f94fffff640 1 -- 172.21.15.133:0/2789977170 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f951000ae20 con 0x7f9518154240 2024-02-21T23:25:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.780+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95180ccfa0 con 0x7f9518154240 2024-02-21T23:25:05.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.887+0000 7f94fffff640 1 -- 172.21.15.133:0/2789977170 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f9510058b60 con 0x7f9518154240 2024-02-21T23:25:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:05.873 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-21T23:25:05.874 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 1 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f950406b490 msgr2=0x7f950406d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f950406b490 0x7f950406d950 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9508036000 tx=0x7f9508008030 comp rx=0 tx=0).stop 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 msgr2=0x7f95180ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 0x7f95180ca650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f951000ca80 tx=0x7f951000cf40 comp rx=0 tx=0).stop 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 shutdown_connections 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f950406b490 0x7f950406d950 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9518154240 0x7f95180ca650 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9518070800 0x7f951815ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 --2- 172.21.15.133:0/2789977170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95180703c0 0x7f951814d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 >> 172.21.15.133:0/2789977170 conn(0x7f9518061b50 msgr2=0x7f9518150330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:05.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 shutdown_connections 2024-02-21T23:25:05.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:05.890+0000 7f951da36640 1 -- 172.21.15.133:0/2789977170 wait complete. 2024-02-21T23:25:07.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:25:07.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.008+0000 7fe352b1e640 1 Processor -- start 2024-02-21T23:25:07.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.008+0000 7fe352b1e640 1 -- start start 2024-02-21T23:25:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.009+0000 7fe352b1e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe34c1542e0 0x7fe34c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.009+0000 7fe352b1e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe34c06bfd0 0x7fe34c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.009+0000 7fe352b1e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34c06c3e0 con 0x7fe34c0aff30 2024-02-21T23:25:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.009+0000 7fe352b1e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34c0b05a0 con 0x7fe34c06bfd0 2024-02-21T23:25:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.009+0000 7fe352b1e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34c06cc40 con 0x7fe34c1542e0 2024-02-21T23:25:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.011+0000 7fe3510bb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe34c06bfd0 0x7fe34c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.012+0000 7fe3510bb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe34c06bfd0 0x7fe34c15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33912/0 (socket says 172.21.15.133:33912) 2024-02-21T23:25:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.012+0000 7fe3510bb640 1 -- 172.21.15.133:0/523163378 learned_addr learned my addr 172.21.15.133:0/523163378 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.015+0000 7fe3510bb640 1 -- 172.21.15.133:0/523163378 >> v2:172.21.15.142:3300/0 conn(0x7fe34c1542e0 msgr2=0x7fe34c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.015+0000 7fe3510bb640 1 --2- 172.21.15.133:0/523163378 >> v2:172.21.15.142:3300/0 conn(0x7fe34c1542e0 0x7fe34c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.015+0000 7fe3510bb640 1 -- 172.21.15.133:0/523163378 >> v1:172.21.15.142:6789/0 conn(0x7fe34c0aff30 legacy=0x7fe34c0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.015+0000 7fe3510bb640 1 -- 172.21.15.133:0/523163378 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c06c680 con 0x7fe34c06bfd0 2024-02-21T23:25:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.018+0000 7fe3510bb640 1 --2- 172.21.15.133:0/523163378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34c06bfd0 0x7fe34c15a180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0c7ee0 tx=0x7fe34402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e79b78a382fa12 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:08.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.027+0000 7fe34b7fe640 1 -- 172.21.15.133:0/523163378 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34402c960 con 0x7fe34c06bfd0 2024-02-21T23:25:08.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.027+0000 7fe34b7fe640 1 -- 172.21.15.133:0/523163378 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34402cb00 con 0x7fe34c06bfd0 2024-02-21T23:25:08.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.027+0000 7fe34b7fe640 1 -- 172.21.15.133:0/523163378 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3440343f0 con 0x7fe34c06bfd0 2024-02-21T23:25:08.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.028+0000 7fe352b1e640 1 -- 172.21.15.133:0/523163378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34c06bfd0 msgr2=0x7fe34c15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:08.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.028+0000 7fe352b1e640 1 --2- 172.21.15.133:0/523163378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34c06bfd0 0x7fe34c15a180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0c7ee0 tx=0x7fe34402bdc0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.028+0000 7fe352b1e640 1 -- 172.21.15.133:0/523163378 shutdown_connections 2024-02-21T23:25:08.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.028+0000 7fe352b1e640 1 --2- 172.21.15.133:0/523163378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34c06bfd0 0x7fe34c15a180 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.028+0000 7fe352b1e640 1 --2- 172.21.15.133:0/523163378 >> v2:172.21.15.142:3300/0 conn(0x7fe34c1542e0 0x7fe34c15e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.028+0000 7fe352b1e640 1 -- 172.21.15.133:0/523163378 >> 172.21.15.133:0/523163378 conn(0x7fe34c06aec0 msgr2=0x7fe34c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:08.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.028+0000 7fe352b1e640 1 -- 172.21.15.133:0/523163378 shutdown_connections 2024-02-21T23:25:08.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.028+0000 7fe352b1e640 1 -- 172.21.15.133:0/523163378 wait complete. 2024-02-21T23:25:08.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.029+0000 7fe352b1e640 1 Processor -- start 2024-02-21T23:25:08.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.029+0000 7fe352b1e640 1 -- start start 2024-02-21T23:25:08.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.030+0000 7fe352b1e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c06bfd0 0x7fe34c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:08.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.030+0000 7fe352b1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe34c0aff30 0x7fe34c1f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:08.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.030+0000 7fe352b1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34c1542e0 0x7fe34c1f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:08.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.030+0000 7fe352b1e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c15daf0 con 0x7fe34c1542e0 2024-02-21T23:25:08.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.030+0000 7fe352b1e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c15d810 con 0x7fe34c06bfd0 2024-02-21T23:25:08.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.030+0000 7fe352b1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c15ddd0 con 0x7fe34c0aff30 2024-02-21T23:25:08.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.031+0000 7fe34bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe34c0aff30 0x7fe34c1f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:08.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.031+0000 7fe3510bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34c1542e0 0x7fe34c1f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:08.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.031+0000 7fe34bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe34c0aff30 0x7fe34c1f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:44132/0 (socket says 172.21.15.133:44132) 2024-02-21T23:25:08.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.031+0000 7fe34bfff640 1 -- 172.21.15.133:0/4233684062 learned_addr learned my addr 172.21.15.133:0/4233684062 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:08.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.035+0000 7fe34bfff640 1 -- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c06bfd0 msgr2=0x7fe34c0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:08.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.035+0000 7fe34bfff640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c06bfd0 0x7fe34c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.035+0000 7fe34bfff640 1 -- 172.21.15.133:0/4233684062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34c1542e0 msgr2=0x7fe34c1f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:08.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.035+0000 7fe34bfff640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34c1542e0 0x7fe34c1f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.035+0000 7fe34bfff640 1 -- 172.21.15.133:0/4233684062 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c158d40 con 0x7fe34c0aff30 2024-02-21T23:25:08.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.038+0000 7fe34bfff640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe34c0aff30 0x7fe34c1f3bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b3e0 tx=0x7fe33800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:08.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.042+0000 7fe3497fa640 1 -- 172.21.15.133:0/4233684062 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338015020 con 0x7fe34c0aff30 2024-02-21T23:25:08.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.042+0000 7fe3497fa640 1 -- 172.21.15.133:0/4233684062 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe338011850 con 0x7fe34c0aff30 2024-02-21T23:25:08.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.042+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34c1595a0 con 0x7fe34c0aff30 2024-02-21T23:25:08.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.042+0000 7fe3497fa640 1 -- 172.21.15.133:0/4233684062 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338011c40 con 0x7fe34c0aff30 2024-02-21T23:25:08.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.042+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34c159b60 con 0x7fe34c0aff30 2024-02-21T23:25:08.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.043+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34c0cd840 con 0x7fe34c0aff30 2024-02-21T23:25:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.066+0000 7fe3497fa640 1 -- 172.21.15.133:0/4233684062 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe338018020 con 0x7fe34c0aff30 2024-02-21T23:25:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.067+0000 7fe3497fa640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe33006b500 0x7fe33006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:08.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.067+0000 7fe3508ba640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe33006b500 0x7fe33006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:08.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.068+0000 7fe3508ba640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe33006b500 0x7fe33006d9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe340009710 tx=0x7fe340006c90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.069+0000 7fe3497fa640 1 -- 172.21.15.133:0/4233684062 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe33805f1d0 con 0x7fe34c0aff30 2024-02-21T23:25:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.454+0000 7fe3497fa640 1 -- 172.21.15.133:0/4233684062 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fe34c0cd840 con 0x7fe34c0aff30 2024-02-21T23:25:08.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.738+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe34c06a130 con 0x7fe34c0aff30 2024-02-21T23:25:08.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.962+0000 7fe3497fa640 1 -- 172.21.15.133:0/4233684062 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fe33801f020 con 0x7fe34c0aff30 2024-02-21T23:25:08.948 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-21T23:25:08.949 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 1 2024-02-21T23:25:08.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.964+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe33006b500 msgr2=0x7fe33006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.964+0000 7fe352b1e640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe33006b500 0x7fe33006d9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe340009710 tx=0x7fe340006c90 comp rx=0 tx=0).stop 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.964+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe34c0aff30 msgr2=0x7fe34c1f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.964+0000 7fe352b1e640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe34c0aff30 0x7fe34c1f3bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b3e0 tx=0x7fe33800b8a0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.965+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 shutdown_connections 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.965+0000 7fe352b1e640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe33006b500 0x7fe33006d9c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.965+0000 7fe352b1e640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34c1542e0 0x7fe34c1f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.965+0000 7fe352b1e640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe34c0aff30 0x7fe34c1f3bb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.965+0000 7fe352b1e640 1 --2- 172.21.15.133:0/4233684062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe34c06bfd0 0x7fe34c0c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.965+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 >> 172.21.15.133:0/4233684062 conn(0x7fe34c06aec0 msgr2=0x7fe34c0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.965+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 shutdown_connections 2024-02-21T23:25:08.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:08.965+0000 7fe352b1e640 1 -- 172.21.15.133:0/4233684062 wait complete. 2024-02-21T23:25:09.847 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-02-21T23:25:09.893 INFO:tasks.ceph.mds.e.smithi133.stdout:starting mds.e at 2024-02-21T23:25:09.911 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-02-21T23:25:09.944 INFO:tasks.ceph.mds.f.smithi142.stdout:starting mds.f at 2024-02-21T23:25:10.033 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-02-21T23:25:10.092 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-02-21T23:25:10.963 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:25:11.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.078+0000 7f9b42b5d640 1 Processor -- start 2024-02-21T23:25:11.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.078+0000 7f9b42b5d640 1 -- start start 2024-02-21T23:25:11.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.079+0000 7f9b42b5d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c0c3dc0 0x7f9b3c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.079+0000 7f9b42b5d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9b3c0c41a0 0x7f9b3c070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.079+0000 7f9b42b5d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c156e90 con 0x7f9b3c150d30 2024-02-21T23:25:11.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.079+0000 7f9b42b5d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c157150 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.079+0000 7f9b42b5d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c157430 con 0x7f9b3c0c3dc0 2024-02-21T23:25:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.080+0000 7f9b408f9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c0c3dc0 0x7f9b3c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.080+0000 7f9b3bfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9b3c0c41a0 0x7f9b3c070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.080+0000 7f9b410fa640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f9b3c150d30 0x7f9b3c15a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57062/0 (socket says 172.21.15.133:57062) 2024-02-21T23:25:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.080+0000 7f9b410fa640 1 -- 172.21.15.133:0/1734316274 learned_addr learned my addr 172.21.15.133:0/1734316274 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:11.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.083+0000 7f9b408f9640 1 -- 172.21.15.133:0/1734316274 >> v2:172.21.15.142:3300/0 conn(0x7f9b3c0c41a0 msgr2=0x7f9b3c070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:11.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.083+0000 7f9b408f9640 1 --2- 172.21.15.133:0/1734316274 >> v2:172.21.15.142:3300/0 conn(0x7f9b3c0c41a0 0x7f9b3c070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.083+0000 7f9b408f9640 1 -- 172.21.15.133:0/1734316274 >> v1:172.21.15.142:6789/0 conn(0x7f9b3c150d30 legacy=0x7f9b3c15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:11.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.083+0000 7f9b408f9640 1 -- 172.21.15.133:0/1734316274 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c157710 con 0x7f9b3c0c3dc0 2024-02-21T23:25:11.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.085+0000 7f9b408f9640 1 --2- 172.21.15.133:0/1734316274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c0c3dc0 0x7f9b3c0cd010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0097a0 tx=0x7f9b2c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46729ffe8502c9ed server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:11.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.089+0000 7f9b3b7fe640 1 -- 172.21.15.133:0/1734316274 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c035020 con 0x7f9b3c0c3dc0 2024-02-21T23:25:11.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.089+0000 7f9b3b7fe640 1 -- 172.21.15.133:0/1734316274 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c031830 con 0x7f9b3c0c3dc0 2024-02-21T23:25:11.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.089+0000 7f9b3b7fe640 1 -- 172.21.15.133:0/1734316274 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c031c00 con 0x7f9b3c0c3dc0 2024-02-21T23:25:11.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.090+0000 7f9b42b5d640 1 -- 172.21.15.133:0/1734316274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c0c3dc0 msgr2=0x7f9b3c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:11.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.090+0000 7f9b42b5d640 1 --2- 172.21.15.133:0/1734316274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c0c3dc0 0x7f9b3c0cd010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0097a0 tx=0x7f9b2c02b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.090+0000 7f9b42b5d640 1 -- 172.21.15.133:0/1734316274 shutdown_connections 2024-02-21T23:25:11.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.090+0000 7f9b42b5d640 1 --2- 172.21.15.133:0/1734316274 >> v2:172.21.15.142:3300/0 conn(0x7f9b3c0c41a0 0x7f9b3c070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.090+0000 7f9b42b5d640 1 --2- 172.21.15.133:0/1734316274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c0c3dc0 0x7f9b3c0cd010 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.090+0000 7f9b42b5d640 1 -- 172.21.15.133:0/1734316274 >> 172.21.15.133:0/1734316274 conn(0x7f9b3c061b50 msgr2=0x7f9b3c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:11.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.090+0000 7f9b42b5d640 1 -- 172.21.15.133:0/1734316274 shutdown_connections 2024-02-21T23:25:11.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.090+0000 7f9b42b5d640 1 -- 172.21.15.133:0/1734316274 wait complete. 2024-02-21T23:25:11.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.091+0000 7f9b42b5d640 1 Processor -- start 2024-02-21T23:25:11.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.091+0000 7f9b42b5d640 1 -- start start 2024-02-21T23:25:11.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.092+0000 7f9b42b5d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c0c3dc0 0x7f9b3c0bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.092+0000 7f9b42b5d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b3c0c41a0 0x7f9b3c0c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.092+0000 7f9b42b5d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c150d30 0x7f9b3c1f1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.092+0000 7f9b42b5d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c070190 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.092+0000 7f9b42b5d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c157cd0 con 0x7f9b3c0c3dc0 2024-02-21T23:25:11.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.092+0000 7f9b42b5d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c070950 con 0x7f9b3c150d30 2024-02-21T23:25:11.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.093+0000 7f9b410fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c150d30 0x7f9b3c1f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.093+0000 7f9b3bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b3c0c41a0 0x7f9b3c0c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.093+0000 7f9b408f9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c0c3dc0 0x7f9b3c0bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.093+0000 7f9b3bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b3c0c41a0 0x7f9b3c0c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33986/0 (socket says 172.21.15.133:33986) 2024-02-21T23:25:11.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.093+0000 7f9b410fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c150d30 0x7f9b3c1f1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:44210/0 (socket says 172.21.15.133:44210) 2024-02-21T23:25:11.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.093+0000 7f9b3bfff640 1 -- 172.21.15.133:0/3060991029 learned_addr learned my addr 172.21.15.133:0/3060991029 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:11.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.097+0000 7f9b3bfff640 1 -- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c150d30 msgr2=0x7f9b3c1f1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:11.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.097+0000 7f9b3bfff640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c150d30 0x7f9b3c1f1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.097+0000 7f9b3bfff640 1 -- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c0c3dc0 msgr2=0x7f9b3c0bdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:11.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.097+0000 7f9b3bfff640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c0c3dc0 0x7f9b3c0bdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.097+0000 7f9b3bfff640 1 -- 172.21.15.133:0/3060991029 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c1ee8f0 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.099+0000 7f9b3bfff640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b3c0c41a0 0x7f9b3c0c7eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000e870 tx=0x7f9b3000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:11.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.102+0000 7f9b397fa640 1 -- 172.21.15.133:0/3060991029 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30017070 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.102+0000 7f9b397fa640 1 -- 172.21.15.133:0/3060991029 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3000c940 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.102+0000 7f9b397fa640 1 -- 172.21.15.133:0/3060991029 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3000cd30 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.102+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c1edd70 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.102+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c1ee330 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.103+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c06a130 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.347+0000 7f9b397fa640 1 -- 172.21.15.133:0/3060991029 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b3001a070 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.347+0000 7f9b397fa640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9b2806b520 0x7f9b2806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.347+0000 7f9b408f9640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9b2806b520 0x7f9b2806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.348+0000 7f9b408f9640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9b2806b520 0x7f9b2806d9e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c036000 tx=0x7f9b2c03c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.360+0000 7f9b397fa640 1 -- 172.21.15.133:0/3060991029 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b3005f310 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.360+0000 7f9b397fa640 1 -- 172.21.15.133:0/3060991029 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f9b30058ff0 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.578+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b3c0cd240 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.600+0000 7f9b397fa640 1 -- 172.21.15.133:0/3060991029 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5292 (secure 0 0 0) 0x7f9b3005d090 con 0x7f9b3c0c41a0 2024-02-21T23:25:11.586 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:11.586 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1319340142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1319340142},{"type":"v1","addr":"172.21.15.133:6837","nonce":1319340142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1740931674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1740931674},{"type":"v1","addr":"172.21.15.142:6837","nonce":1740931674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3712864993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3712864993},{"type":"v1","addr":"172.21.15.142:6839","nonce":3712864993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2928905924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2928905924},{"type":"v1","addr":"172.21.15.133:6833","nonce":2928905924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3461010406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3461010406},{"type":"v1","addr":"172.21.15.133:6835","nonce":3461010406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1527124199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1527124199},{"type":"v1","addr":"172.21.15.142:6835","nonce":1527124199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-21T23:25:11.586 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 3 2024-02-21T23:25:11.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9b2806b520 msgr2=0x7f9b2806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:11.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9b2806b520 0x7f9b2806d9e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c036000 tx=0x7f9b2c03c040 comp rx=0 tx=0).stop 2024-02-21T23:25:11.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b3c0c41a0 msgr2=0x7f9b3c0c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:11.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b3c0c41a0 0x7f9b3c0c7eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000e870 tx=0x7f9b3000ed30 comp rx=0 tx=0).stop 2024-02-21T23:25:11.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 shutdown_connections 2024-02-21T23:25:11.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9b2806b520 0x7f9b2806d9e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b3c150d30 0x7f9b3c1f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b3c0c41a0 0x7f9b3c0c7eb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 --2- 172.21.15.133:0/3060991029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b3c0c3dc0 0x7f9b3c0bdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 >> 172.21.15.133:0/3060991029 conn(0x7f9b3c061b50 msgr2=0x7f9b3c158370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:11.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 shutdown_connections 2024-02-21T23:25:11.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.602+0000 7f9b42b5d640 1 -- 172.21.15.133:0/3060991029 wait complete. 2024-02-21T23:25:11.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:11.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.719+0000 7f3ef5f15640 1 Processor -- start 2024-02-21T23:25:11.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.720+0000 7f3ef5f15640 1 -- start start 2024-02-21T23:25:11.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.721+0000 7f3ef5f15640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3ef0154600 0x7f3ef00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.721+0000 7f3ef5f15640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef015ab40 con 0x7f3ef00c6f50 2024-02-21T23:25:11.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.721+0000 7f3ef5f15640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef015ae00 con 0x7f3ef00c7c10 2024-02-21T23:25:11.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.721+0000 7f3ef5f15640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef00c74a0 con 0x7f3ef0154600 2024-02-21T23:25:11.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.722+0000 7f3eeeffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3ef00c6f50 0x7f3ef00ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51954/0 (socket says 172.21.15.133:51954) 2024-02-21T23:25:11.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.722+0000 7f3eeeffd640 1 -- 172.21.15.133:0/2570774603 learned_addr learned my addr 172.21.15.133:0/2570774603 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:11.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.724+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137928248 0 0) 0x7f3ef015ab40 con 0x7f3ef00c6f50 2024-02-21T23:25:11.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.724+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed8003520 con 0x7f3ef00c6f50 2024-02-21T23:25:11.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4085559509 0 0) 0x7f3ed8003520 con 0x7f3ef00c6f50 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 >> v2:172.21.15.142:3300/0 conn(0x7f3ef0154600 msgr2=0x7f3ef00cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 --2- 172.21.15.133:0/2570774603 >> v2:172.21.15.142:3300/0 conn(0x7f3ef0154600 0x7f3ef00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 >> v1:172.21.15.142:6789/0 conn(0x7f3ef00c7c10 legacy=0x7f3ef00c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00cb580 con 0x7f3ef00c6f50 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 797941756 0 0) 0x7f3ee4001670 con 0x7f3ef00c6f50 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 >> v1:172.21.15.133:6789/0 conn(0x7f3ef00c6f50 legacy=0x7f3ef00ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 --2- 172.21.15.133:0/2570774603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed8003a00 0x7f3ed8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 --2- 172.21.15.133:0/2570774603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed8004800 0x7f3ed8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 --2- 172.21.15.133:0/2570774603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ed80052e0 0x7f3ed800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00c74a0 con 0x7f3ed80052e0 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed800c230 con 0x7f3ed8004800 2024-02-21T23:25:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.728+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed800c430 con 0x7f3ed8003a00 2024-02-21T23:25:11.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.732+0000 7f3eeffff640 1 --2- 172.21.15.133:0/2570774603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ed80052e0 0x7f3ed800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.734+0000 7f3eeffff640 1 -- 172.21.15.133:0/2570774603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed8003a00 msgr2=0x7f3ed8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:11.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.734+0000 7f3eeffff640 1 --2- 172.21.15.133:0/2570774603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed8003a00 0x7f3ed8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.734+0000 7f3eeffff640 1 -- 172.21.15.133:0/2570774603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed8004800 msgr2=0x7f3ed8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:11.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.734+0000 7f3eeffff640 1 --2- 172.21.15.133:0/2570774603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed8004800 0x7f3ed8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.734+0000 7f3eeffff640 1 -- 172.21.15.133:0/2570774603 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed8003520 con 0x7f3ed80052e0 2024-02-21T23:25:11.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.735+0000 7f3eeffff640 1 -- 172.21.15.133:0/2570774603 reap_dead start 2024-02-21T23:25:11.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.737+0000 7f3eeffff640 1 --2- 172.21.15.133:0/2570774603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ed80052e0 0x7f3ed800b9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8001d20 tx=0x7f3ed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41d24c19a3efe16 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:11.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.741+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8001920 con 0x7f3ed80052e0 2024-02-21T23:25:11.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.741+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8003b90 con 0x7f3ed80052e0 2024-02-21T23:25:11.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.741+0000 7f3eee7fc640 1 -- 172.21.15.133:0/2570774603 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8005070 con 0x7f3ed80052e0 2024-02-21T23:25:11.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.742+0000 7f3ef5f15640 1 -- 172.21.15.133:0/2570774603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ed80052e0 msgr2=0x7f3ed800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:11.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.742+0000 7f3ef5f15640 1 --2- 172.21.15.133:0/2570774603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ed80052e0 0x7f3ed800b9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8001d20 tx=0x7f3ed8005830 comp rx=0 tx=0).stop 2024-02-21T23:25:11.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.742+0000 7f3ef5f15640 1 -- 172.21.15.133:0/2570774603 shutdown_connections 2024-02-21T23:25:11.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.742+0000 7f3ef5f15640 1 --2- 172.21.15.133:0/2570774603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ed80052e0 0x7f3ed800b9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.742+0000 7f3ef5f15640 1 -- 172.21.15.133:0/2570774603 >> 172.21.15.133:0/2570774603 conn(0x7f3ef0062000 msgr2=0x7f3ef00cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:11.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.742+0000 7f3ef5f15640 1 -- 172.21.15.133:0/2570774603 shutdown_connections 2024-02-21T23:25:11.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.742+0000 7f3ef5f15640 1 -- 172.21.15.133:0/2570774603 wait complete. 2024-02-21T23:25:11.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.744+0000 7f3ef5f15640 1 Processor -- start 2024-02-21T23:25:11.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.744+0000 7f3ef5f15640 1 -- start start 2024-02-21T23:25:11.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.745+0000 7f3ef5f15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed80052e0 0x7f3ef014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.745+0000 7f3ef5f15640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ef014c8c0 0x7f3ef0153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.745+0000 7f3ef5f15640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef0151900 0x7f3ef0158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.745+0000 7f3ef5f15640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00b5720 con 0x7f3ef0151900 2024-02-21T23:25:11.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.745+0000 7f3ef5f15640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00c7780 con 0x7f3ed80052e0 2024-02-21T23:25:11.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.745+0000 7f3ef5f15640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00b58a0 con 0x7f3ef014c8c0 2024-02-21T23:25:11.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.746+0000 7f3eeeffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ef014c8c0 0x7f3ef0153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.746+0000 7f3eeffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef0151900 0x7f3ef0158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.746+0000 7f3eeeffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ef014c8c0 0x7f3ef0153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:33450/0 (socket says 172.21.15.133:33450) 2024-02-21T23:25:11.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.746+0000 7f3eeeffd640 1 -- 172.21.15.133:0/1117347380 learned_addr learned my addr 172.21.15.133:0/1117347380 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:11.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.751+0000 7f3eeffff640 1 -- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ef014c8c0 msgr2=0x7f3ef0153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:11.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.751+0000 7f3eeffff640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ef014c8c0 0x7f3ef0153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.751+0000 7f3eeffff640 1 -- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed80052e0 msgr2=0x7f3ef014ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:11.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.751+0000 7f3eeffff640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed80052e0 0x7f3ef014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:11.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.751+0000 7f3eeffff640 1 -- 172.21.15.133:0/1117347380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef014b2e0 con 0x7f3ef0151900 2024-02-21T23:25:11.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.754+0000 7f3eeffff640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef0151900 0x7f3ef0158a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ef008da40 tx=0x7f3ee8001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.758+0000 7f3ed7fff640 1 -- 172.21.15.133:0/1117347380 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8005540 con 0x7f3ef0151900 2024-02-21T23:25:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.758+0000 7f3ed7fff640 1 -- 172.21.15.133:0/1117347380 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8040070 con 0x7f3ef0151900 2024-02-21T23:25:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.758+0000 7f3ed7fff640 1 -- 172.21.15.133:0/1117347380 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee803a460 con 0x7f3ef0151900 2024-02-21T23:25:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.758+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef014b5c0 con 0x7f3ef0151900 2024-02-21T23:25:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.759+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef014bb80 con 0x7f3ef0151900 2024-02-21T23:25:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.760+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef006a130 con 0x7f3ef0151900 2024-02-21T23:25:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.780+0000 7f3ed7fff640 1 -- 172.21.15.133:0/1117347380 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ee803a600 con 0x7f3ef0151900 2024-02-21T23:25:11.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.780+0000 7f3ed7fff640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3edc06b550 0x7f3edc06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:11.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.780+0000 7f3eef7fe640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3edc06b550 0x7f3edc06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:11.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:11.781+0000 7f3eef7fe640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3edc06b550 0x7f3edc06da10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009710 tx=0x7f3ee0006d00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.025+0000 7f3ed7fff640 1 -- 172.21.15.133:0/1117347380 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3ee80786b0 con 0x7f3ef0151900 2024-02-21T23:25:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.025+0000 7f3ed7fff640 1 -- 172.21.15.133:0/1117347380 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f3ee807ad10 con 0x7f3ef0151900 2024-02-21T23:25:12.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.198+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ef015ab40 con 0x7f3ef0151900 2024-02-21T23:25:12.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.222+0000 7f3ed7fff640 1 -- 172.21.15.133:0/1117347380 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12932 (secure 0 0 0) 0x7f3ee807a610 con 0x7f3ef0151900 2024-02-21T23:25:12.208 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:12.208 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":16,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:06.505044+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:12.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3edc06b550 msgr2=0x7f3edc06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3edc06b550 0x7f3edc06da10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009710 tx=0x7f3ee0006d00 comp rx=0 tx=0).stop 2024-02-21T23:25:12.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef0151900 msgr2=0x7f3ef0158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef0151900 0x7f3ef0158a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ef008da40 tx=0x7f3ee8001330 comp rx=0 tx=0).stop 2024-02-21T23:25:12.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 shutdown_connections 2024-02-21T23:25:12.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3edc06b550 0x7f3edc06da10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef0151900 0x7f3ef0158a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ef014c8c0 0x7f3ef0153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 --2- 172.21.15.133:0/1117347380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed80052e0 0x7f3ef014ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 >> 172.21.15.133:0/1117347380 conn(0x7f3ef0062000 msgr2=0x7f3ef00cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:12.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 shutdown_connections 2024-02-21T23:25:12.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.224+0000 7f3ef5f15640 1 -- 172.21.15.133:0/1117347380 wait complete. 2024-02-21T23:25:12.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-21T23:25:02.902698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-21T23:25:12.222 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-21T23:25:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.339+0000 7f34ca31e640 1 Processor -- start 2024-02-21T23:25:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.339+0000 7f34ca31e640 1 -- start start 2024-02-21T23:25:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.339+0000 7f34ca31e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4154450 0x7f34c406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.340+0000 7f34ca31e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f34c415a290 0x7f34c40c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.340+0000 7f34ca31e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c415a720 con 0x7f34c406ff00 2024-02-21T23:25:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.340+0000 7f34ca31e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c406c2f0 con 0x7f34c415a290 2024-02-21T23:25:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.340+0000 7f34ca31e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c415a9e0 con 0x7f34c4154450 2024-02-21T23:25:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.340+0000 7f34c37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f34c415a290 0x7f34c40c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.340+0000 7f34c3fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4154450 0x7f34c406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.340+0000 7f34c88bb640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f34c406ff00 0x7f34c40c3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:45618/0 (socket says 172.21.15.133:45618) 2024-02-21T23:25:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.340+0000 7f34c88bb640 1 -- 172.21.15.133:0/3978622208 learned_addr learned my addr 172.21.15.133:0/3978622208 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.343+0000 7f34c2ffd640 1 -- 172.21.15.133:0/3978622208 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781578414 0 0) 0x7f34c415a720 con 0x7f34c406ff00 2024-02-21T23:25:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.343+0000 7f34c2ffd640 1 -- 172.21.15.133:0/3978622208 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34ac003520 con 0x7f34c406ff00 2024-02-21T23:25:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.343+0000 7f34c37fe640 1 -- 172.21.15.133:0/3978622208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4154450 msgr2=0x7f34c406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.343+0000 7f34c37fe640 1 --2- 172.21.15.133:0/3978622208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4154450 0x7f34c406f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.343+0000 7f34c37fe640 1 -- 172.21.15.133:0/3978622208 >> v1:172.21.15.142:6789/0 conn(0x7f34c406ff00 legacy=0x7f34c40c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.343+0000 7f34c37fe640 1 -- 172.21.15.133:0/3978622208 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c415acc0 con 0x7f34c415a290 2024-02-21T23:25:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.345+0000 7f34c37fe640 1 --2- 172.21.15.133:0/3978622208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c415a290 0x7f34c40c8230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f34b80097a0 tx=0x7f34b802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1c4b6be6c97106 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.348+0000 7f34c2ffd640 1 -- 172.21.15.133:0/3978622208 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8034070 con 0x7f34c415a290 2024-02-21T23:25:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.348+0000 7f34c2ffd640 1 -- 172.21.15.133:0/3978622208 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b8030610 con 0x7f34c415a290 2024-02-21T23:25:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.348+0000 7f34c2ffd640 1 -- 172.21.15.133:0/3978622208 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b80309e0 con 0x7f34c415a290 2024-02-21T23:25:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.349+0000 7f34ca31e640 1 -- 172.21.15.133:0/3978622208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c415a290 msgr2=0x7f34c40c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.349+0000 7f34ca31e640 1 --2- 172.21.15.133:0/3978622208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c415a290 0x7f34c40c8230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f34b80097a0 tx=0x7f34b802b960 comp rx=0 tx=0).stop 2024-02-21T23:25:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.349+0000 7f34ca31e640 1 -- 172.21.15.133:0/3978622208 shutdown_connections 2024-02-21T23:25:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.349+0000 7f34ca31e640 1 --2- 172.21.15.133:0/3978622208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c415a290 0x7f34c40c8230 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.349+0000 7f34ca31e640 1 --2- 172.21.15.133:0/3978622208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4154450 0x7f34c406f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.349+0000 7f34ca31e640 1 -- 172.21.15.133:0/3978622208 >> 172.21.15.133:0/3978622208 conn(0x7f34c4070ec0 msgr2=0x7f34c40712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.349+0000 7f34ca31e640 1 -- 172.21.15.133:0/3978622208 shutdown_connections 2024-02-21T23:25:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.349+0000 7f34ca31e640 1 -- 172.21.15.133:0/3978622208 wait complete. 2024-02-21T23:25:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.350+0000 7f34ca31e640 1 Processor -- start 2024-02-21T23:25:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.350+0000 7f34ca31e640 1 -- start start 2024-02-21T23:25:12.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.351+0000 7f34ca31e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c406ff00 0x7f34c4152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.351+0000 7f34ca31e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c4154450 0x7f34c4144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.351+0000 7f34ca31e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4146b20 0x7f34c41f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.351+0000 7f34ca31e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c40c3d50 con 0x7f34c4154450 2024-02-21T23:25:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.351+0000 7f34ca31e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c406f4a0 con 0x7f34c406ff00 2024-02-21T23:25:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.351+0000 7f34ca31e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c40c3ed0 con 0x7f34c4146b20 2024-02-21T23:25:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.351+0000 7f34c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c4154450 0x7f34c4144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.352+0000 7f34c3fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c406ff00 0x7f34c4152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.352+0000 7f34c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c4154450 0x7f34c4144770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51192/0 (socket says 172.21.15.133:51192) 2024-02-21T23:25:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.352+0000 7f34c37fe640 1 -- 172.21.15.133:0/73174811 learned_addr learned my addr 172.21.15.133:0/73174811 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.352+0000 7f34c88bb640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4146b20 0x7f34c41f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.355+0000 7f34c37fe640 1 -- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4146b20 msgr2=0x7f34c41f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.355+0000 7f34c37fe640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4146b20 0x7f34c41f3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.355+0000 7f34c37fe640 1 -- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c406ff00 msgr2=0x7f34c4152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.355+0000 7f34c37fe640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c406ff00 0x7f34c4152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.355+0000 7f34c37fe640 1 -- 172.21.15.133:0/73174811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c4146090 con 0x7f34c4154450 2024-02-21T23:25:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.357+0000 7f34c37fe640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c4154450 0x7f34c4144770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34b802bf70 tx=0x7f34b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.361+0000 7f34c0ff9640 1 -- 172.21.15.133:0/73174811 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8034040 con 0x7f34c4154450 2024-02-21T23:25:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.361+0000 7f34c0ff9640 1 -- 172.21.15.133:0/73174811 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b803aa20 con 0x7f34c4154450 2024-02-21T23:25:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.361+0000 7f34c0ff9640 1 -- 172.21.15.133:0/73174811 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b803ae10 con 0x7f34c4154450 2024-02-21T23:25:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.361+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c4145510 con 0x7f34c4154450 2024-02-21T23:25:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.361+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c4145ad0 con 0x7f34c4154450 2024-02-21T23:25:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.362+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c40c2d60 con 0x7f34c4154450 2024-02-21T23:25:12.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.622+0000 7f34c0ff9640 1 -- 172.21.15.133:0/73174811 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34b804b060 con 0x7f34c4154450 2024-02-21T23:25:12.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.622+0000 7f34c0ff9640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f34b006b520 0x7f34b006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.622+0000 7f34c3fff640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f34b006b520 0x7f34b006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.623+0000 7f34c3fff640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f34b006b520 0x7f34b006d9e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009b30 tx=0x7f34b4009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.636+0000 7f34c0ff9640 1 -- 172.21.15.133:0/73174811 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f34b8080c50 con 0x7f34c4154450 2024-02-21T23:25:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.637+0000 7f34c0ff9640 1 -- 172.21.15.133:0/73174811 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f34c40c2d60 con 0x7f34c4154450 2024-02-21T23:25:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.828+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34c406a130 con 0x7f34c4154450 2024-02-21T23:25:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.833+0000 7f34c0ff9640 1 -- 172.21.15.133:0/73174811 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34b803f070 con 0x7f34c4154450 2024-02-21T23:25:12.818 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f34b006b520 msgr2=0x7f34b006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f34b006b520 0x7f34b006d9e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009b30 tx=0x7f34b4009110 comp rx=0 tx=0).stop 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c4154450 msgr2=0x7f34c4144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c4154450 0x7f34c4144770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34b802bf70 tx=0x7f34b80046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 shutdown_connections 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f34b006b520 0x7f34b006d9e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c4146b20 0x7f34c41f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34c4154450 0x7f34c4144770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 --2- 172.21.15.133:0/73174811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c406ff00 0x7f34c4152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 >> 172.21.15.133:0/73174811 conn(0x7f34c4070ec0 msgr2=0x7f34c406d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 shutdown_connections 2024-02-21T23:25:12.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.835+0000 7f34ca31e640 1 -- 172.21.15.133:0/73174811 wait complete. 2024-02-21T23:25:12.832 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-21T23:25:12.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.947+0000 7ff4bfcd5640 1 Processor -- start 2024-02-21T23:25:12.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.947+0000 7ff4bfcd5640 1 -- start start 2024-02-21T23:25:12.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.948+0000 7ff4bfcd5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c7790 0x7ff4b815d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.948+0000 7ff4bfcd5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff4b80c9830 0x7ff4b815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.948+0000 7ff4bfcd5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b8073210 con 0x7ff4b80c9450 2024-02-21T23:25:12.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.948+0000 7ff4bfcd5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b815a280 con 0x7ff4b80c9830 2024-02-21T23:25:12.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.948+0000 7ff4bfcd5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b815a560 con 0x7ff4b80c7790 2024-02-21T23:25:12.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.949+0000 7ff4bd270640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c7790 0x7ff4b815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.949+0000 7ff4be272640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff4b80c9830 0x7ff4b815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.949+0000 7ff4bda71640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7ff4b80c9450 0x7ff4b8072a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:45620/0 (socket says 172.21.15.133:45620) 2024-02-21T23:25:12.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.949+0000 7ff4bda71640 1 -- 172.21.15.133:0/833288469 learned_addr learned my addr 172.21.15.133:0/833288469 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:12.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.952+0000 7ff4bd270640 1 -- 172.21.15.133:0/833288469 >> v2:172.21.15.142:3300/0 conn(0x7ff4b80c9830 msgr2=0x7ff4b815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:12.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.952+0000 7ff4bd270640 1 --2- 172.21.15.133:0/833288469 >> v2:172.21.15.142:3300/0 conn(0x7ff4b80c9830 0x7ff4b815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.952+0000 7ff4bd270640 1 -- 172.21.15.133:0/833288469 >> v1:172.21.15.142:6789/0 conn(0x7ff4b80c9450 legacy=0x7ff4b8072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.952+0000 7ff4bd270640 1 -- 172.21.15.133:0/833288469 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b815a840 con 0x7ff4b80c7790 2024-02-21T23:25:12.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.954+0000 7ff4bd270640 1 --2- 172.21.15.133:0/833288469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c7790 0x7ff4b815d760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097a0 tx=0x7ff4a802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84287810e925795f server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:12.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.958+0000 7ff4bca6f640 1 -- 172.21.15.133:0/833288469 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8034070 con 0x7ff4b80c7790 2024-02-21T23:25:12.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.958+0000 7ff4bca6f640 1 -- 172.21.15.133:0/833288469 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a8030610 con 0x7ff4b80c7790 2024-02-21T23:25:12.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.958+0000 7ff4bca6f640 1 -- 172.21.15.133:0/833288469 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80309e0 con 0x7ff4b80c7790 2024-02-21T23:25:12.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.958+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/833288469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c7790 msgr2=0x7ff4b815d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.959+0000 7ff4bfcd5640 1 --2- 172.21.15.133:0/833288469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c7790 0x7ff4b815d760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097a0 tx=0x7ff4a802b960 comp rx=0 tx=0).stop 2024-02-21T23:25:12.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.959+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/833288469 shutdown_connections 2024-02-21T23:25:12.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.959+0000 7ff4bfcd5640 1 --2- 172.21.15.133:0/833288469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c7790 0x7ff4b815d760 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.959+0000 7ff4bfcd5640 1 --2- 172.21.15.133:0/833288469 >> v2:172.21.15.142:3300/0 conn(0x7ff4b80c9830 0x7ff4b815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.959+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/833288469 >> 172.21.15.133:0/833288469 conn(0x7ff4b806bfd0 msgr2=0x7ff4b806c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:12.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.959+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/833288469 shutdown_connections 2024-02-21T23:25:12.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.959+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/833288469 wait complete. 2024-02-21T23:25:12.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.960+0000 7ff4bfcd5640 1 Processor -- start 2024-02-21T23:25:12.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.960+0000 7ff4bfcd5640 1 -- start start 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.960+0000 7ff4bfcd5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b80c7790 0x7ff4b80cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.960+0000 7ff4bfcd5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4b80c9450 0x7ff4b8150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.960+0000 7ff4bfcd5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c9830 0x7ff4b8148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.960+0000 7ff4bfcd5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8070b30 con 0x7ff4b80c7790 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.960+0000 7ff4bfcd5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80709b0 con 0x7ff4b80c9450 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.960+0000 7ff4bfcd5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81f3eb0 con 0x7ff4b80c9830 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.961+0000 7ff4bd270640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4b80c9450 0x7ff4b8150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.961+0000 7ff4bda71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b80c7790 0x7ff4b80cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.961+0000 7ff4be272640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c9830 0x7ff4b8148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.961+0000 7ff4bd270640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4b80c9450 0x7ff4b8150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:44602/0 (socket says 172.21.15.133:44602) 2024-02-21T23:25:12.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.961+0000 7ff4bd270640 1 -- 172.21.15.133:0/2887389766 learned_addr learned my addr 172.21.15.133:0/2887389766 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:12.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.965+0000 7ff4bd270640 1 -- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c9830 msgr2=0x7ff4b8148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.965+0000 7ff4bd270640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c9830 0x7ff4b8148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.965+0000 7ff4bd270640 1 -- 172.21.15.133:0/2887389766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b80c7790 msgr2=0x7ff4b80cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:12.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.965+0000 7ff4bd270640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b80c7790 0x7ff4b80cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:12.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.965+0000 7ff4bd270640 1 -- 172.21.15.133:0/2887389766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80c8c50 con 0x7ff4b80c9450 2024-02-21T23:25:12.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.967+0000 7ff4bd270640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4b80c9450 0x7ff4b8150e60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4a802bf70 tx=0x7ff4a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:12.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.971+0000 7ff4ae7fc640 1 -- 172.21.15.133:0/2887389766 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8034040 con 0x7ff4b80c9450 2024-02-21T23:25:12.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.971+0000 7ff4ae7fc640 1 -- 172.21.15.133:0/2887389766 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a803ac40 con 0x7ff4b80c9450 2024-02-21T23:25:12.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.971+0000 7ff4ae7fc640 1 -- 172.21.15.133:0/2887389766 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80443f0 con 0x7ff4b80c9450 2024-02-21T23:25:12.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.971+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b80c6d80 con 0x7ff4b80c9450 2024-02-21T23:25:12.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.971+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b81f33d0 con 0x7ff4b80c9450 2024-02-21T23:25:12.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:12.972+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b806a130 con 0x7ff4b80c9450 2024-02-21T23:25:13.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.216+0000 7ff4ae7fc640 1 -- 172.21.15.133:0/2887389766 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4a804b060 con 0x7ff4b80c9450 2024-02-21T23:25:13.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.216+0000 7ff4ae7fc640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff49c06b520 0x7ff49c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:13.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.216+0000 7ff4bda71640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff49c06b520 0x7ff49c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:13.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.217+0000 7ff4bda71640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff49c06b520 0x7ff49c06d9e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40012e0 tx=0x7ff4b4001a30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:13.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.218+0000 7ff4ae7fc640 1 -- 172.21.15.133:0/2887389766 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff4a8080ce0 con 0x7ff4b80c9450 2024-02-21T23:25:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.228+0000 7ff4ae7fc640 1 -- 172.21.15.133:0/2887389766 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7ff4a803f070 con 0x7ff4b80c9450 2024-02-21T23:25:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.403+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff4b815d000 con 0x7ff4b80c9450 2024-02-21T23:25:13.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.412+0000 7ff4ae7fc640 1 -- 172.21.15.133:0/2887389766 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff4a807a9c0 con 0x7ff4b80c9450 2024-02-21T23:25:13.398 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:13.398 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-02-21T23:25:13.398 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-02-21T23:25:13.398 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-02-21T23:25:13.398 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-02-21T23:25:13.398 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-02-21T23:25:13.398 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-02-21T23:25:13.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff49c06b520 msgr2=0x7ff49c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:13.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff49c06b520 0x7ff49c06d9e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40012e0 tx=0x7ff4b4001a30 comp rx=0 tx=0).stop 2024-02-21T23:25:13.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4b80c9450 msgr2=0x7ff4b8150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:13.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4b80c9450 0x7ff4b8150e60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4a802bf70 tx=0x7ff4a80046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 shutdown_connections 2024-02-21T23:25:13.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff49c06b520 0x7ff49c06d9e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4b80c9830 0x7ff4b8148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4b80c9450 0x7ff4b8150e60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 --2- 172.21.15.133:0/2887389766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b80c7790 0x7ff4b80cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 >> 172.21.15.133:0/2887389766 conn(0x7ff4b806bfd0 msgr2=0x7ff4b8072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:13.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 shutdown_connections 2024-02-21T23:25:13.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.414+0000 7ff4bfcd5640 1 -- 172.21.15.133:0/2887389766 wait complete. 2024-02-21T23:25:13.410 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-21T23:25:13.411 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-21T23:25:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.529+0000 7fb3682de640 1 Processor -- start 2024-02-21T23:25:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.529+0000 7fb3682de640 1 -- start start 2024-02-21T23:25:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.530+0000 7fb3682de640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb360154450 0x7fb36006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.530+0000 7fb3682de640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb36015a290 0x7fb3600c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.530+0000 7fb3682de640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36015a720 con 0x7fb36006ff00 2024-02-21T23:25:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.530+0000 7fb3682de640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36006c2f0 con 0x7fb36015a290 2024-02-21T23:25:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.530+0000 7fb3682de640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36015a9e0 con 0x7fb360154450 2024-02-21T23:25:13.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.530+0000 7fb365879640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb36015a290 0x7fb3600c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:13.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.531+0000 7fb365879640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb36015a290 0x7fb3600c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51214/0 (socket says 172.21.15.133:51214) 2024-02-21T23:25:13.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.531+0000 7fb365879640 1 -- 172.21.15.133:0/2712938590 learned_addr learned my addr 172.21.15.133:0/2712938590 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:13.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.531+0000 7fb36607a640 1 --2- 172.21.15.133:0/2712938590 >> v2:172.21.15.142:3300/0 conn(0x7fb360154450 0x7fb36006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.533+0000 7fb365879640 1 -- 172.21.15.133:0/2712938590 >> v2:172.21.15.142:3300/0 conn(0x7fb360154450 msgr2=0x7fb36006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.533+0000 7fb365879640 1 --2- 172.21.15.133:0/2712938590 >> v2:172.21.15.142:3300/0 conn(0x7fb360154450 0x7fb36006f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.533+0000 7fb365879640 1 -- 172.21.15.133:0/2712938590 >> v1:172.21.15.142:6789/0 conn(0x7fb36006ff00 legacy=0x7fb3600c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.533+0000 7fb365879640 1 -- 172.21.15.133:0/2712938590 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36015acc0 con 0x7fb36015a290 2024-02-21T23:25:13.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.535+0000 7fb365879640 1 --2- 172.21.15.133:0/2712938590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb36015a290 0x7fb3600c8230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb36006e1b0 tx=0x7fb35002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2540ace6e0ff6c server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:13.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.539+0000 7fb365078640 1 -- 172.21.15.133:0/2712938590 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35002f850 con 0x7fb36015a290 2024-02-21T23:25:13.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.539+0000 7fb365078640 1 -- 172.21.15.133:0/2712938590 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35002f9f0 con 0x7fb36015a290 2024-02-21T23:25:13.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.539+0000 7fb365078640 1 -- 172.21.15.133:0/2712938590 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35002fdc0 con 0x7fb36015a290 2024-02-21T23:25:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.540+0000 7fb3682de640 1 -- 172.21.15.133:0/2712938590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb36015a290 msgr2=0x7fb3600c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.540+0000 7fb3682de640 1 --2- 172.21.15.133:0/2712938590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb36015a290 0x7fb3600c8230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb36006e1b0 tx=0x7fb35002ba30 comp rx=0 tx=0).stop 2024-02-21T23:25:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.540+0000 7fb3682de640 1 -- 172.21.15.133:0/2712938590 shutdown_connections 2024-02-21T23:25:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.540+0000 7fb3682de640 1 --2- 172.21.15.133:0/2712938590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb36015a290 0x7fb3600c8230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.540+0000 7fb3682de640 1 --2- 172.21.15.133:0/2712938590 >> v2:172.21.15.142:3300/0 conn(0x7fb360154450 0x7fb36006f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.540+0000 7fb3682de640 1 -- 172.21.15.133:0/2712938590 >> 172.21.15.133:0/2712938590 conn(0x7fb360070ec0 msgr2=0x7fb3600712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.540+0000 7fb3682de640 1 -- 172.21.15.133:0/2712938590 shutdown_connections 2024-02-21T23:25:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.540+0000 7fb3682de640 1 -- 172.21.15.133:0/2712938590 wait complete. 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.541+0000 7fb3682de640 1 Processor -- start 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.541+0000 7fb3682de640 1 -- start start 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.542+0000 7fb3682de640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb36006ff00 0x7fb3600c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.542+0000 7fb3682de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb360154450 0x7fb3600ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.542+0000 7fb3682de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3600ca690 0x7fb3601eff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.542+0000 7fb3682de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600c3d50 con 0x7fb360154450 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.542+0000 7fb3682de640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36006f4a0 con 0x7fb36006ff00 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.542+0000 7fb3682de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600c3ed0 con 0x7fb3600ca690 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.542+0000 7fb365879640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb360154450 0x7fb3600ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.542+0000 7fb36607a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb36006ff00 0x7fb3600c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.542+0000 7fb36687b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3600ca690 0x7fb3601eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.543+0000 7fb365879640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb360154450 0x7fb3600ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51224/0 (socket says 172.21.15.133:51224) 2024-02-21T23:25:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.543+0000 7fb365879640 1 -- 172.21.15.133:0/173152905 learned_addr learned my addr 172.21.15.133:0/173152905 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.546+0000 7fb365879640 1 -- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3600ca690 msgr2=0x7fb3601eff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.546+0000 7fb365879640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3600ca690 0x7fb3601eff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.546+0000 7fb365879640 1 -- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb36006ff00 msgr2=0x7fb3600c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.546+0000 7fb365879640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb36006ff00 0x7fb3600c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.546+0000 7fb365879640 1 -- 172.21.15.133:0/173152905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3600c7c30 con 0x7fb360154450 2024-02-21T23:25:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.546+0000 7fb36607a640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb36006ff00 0x7fb3600c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:25:13.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.548+0000 7fb365879640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb360154450 0x7fb3600ca150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3500090b0 tx=0x7fb3500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:13.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.552+0000 7fb356ffd640 1 -- 172.21.15.133:0/173152905 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350037c80 con 0x7fb360154450 2024-02-21T23:25:13.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.552+0000 7fb356ffd640 1 -- 172.21.15.133:0/173152905 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb350047070 con 0x7fb360154450 2024-02-21T23:25:13.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.552+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3600c6e00 con 0x7fb360154450 2024-02-21T23:25:13.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.552+0000 7fb356ffd640 1 -- 172.21.15.133:0/173152905 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500426c0 con 0x7fb360154450 2024-02-21T23:25:13.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.552+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3600c4560 con 0x7fb360154450 2024-02-21T23:25:13.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.553+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36006a130 con 0x7fb360154450 2024-02-21T23:25:13.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.571+0000 7fb356ffd640 1 -- 172.21.15.133:0/173152905 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb350040050 con 0x7fb360154450 2024-02-21T23:25:13.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.572+0000 7fb356ffd640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb34406b520 0x7fb34406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:13.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.572+0000 7fb36607a640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb34406b520 0x7fb34406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:13.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.572+0000 7fb36607a640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb34406b520 0x7fb34406d9e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009f20 tx=0x7fb35c009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:13.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.574+0000 7fb356ffd640 1 -- 172.21.15.133:0/173152905 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb35007a0f0 con 0x7fb360154450 2024-02-21T23:25:13.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.809+0000 7fb356ffd640 1 -- 172.21.15.133:0/173152905 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fb350033030 con 0x7fb360154450 2024-02-21T23:25:13.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:13.986+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb360155d60 con 0x7fb360154450 2024-02-21T23:25:14.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:14.999+0000 7fb356ffd640 1 -- 172.21.15.133:0/173152905 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb350037e20 con 0x7fb360154450 2024-02-21T23:25:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-02-21T23:25:14.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb34406b520 msgr2=0x7fb34406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb34406b520 0x7fb34406d9e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009f20 tx=0x7fb35c009110 comp rx=0 tx=0).stop 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb360154450 msgr2=0x7fb3600ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb360154450 0x7fb3600ca150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3500090b0 tx=0x7fb3500046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 shutdown_connections 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb34406b520 0x7fb34406d9e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3600ca690 0x7fb3601eff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb360154450 0x7fb3600ca150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 --2- 172.21.15.133:0/173152905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb36006ff00 0x7fb3600c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 >> 172.21.15.133:0/173152905 conn(0x7fb360070ec0 msgr2=0x7fb3600ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 shutdown_connections 2024-02-21T23:25:14.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.002+0000 7fb3682de640 1 -- 172.21.15.133:0/173152905 wait complete. 2024-02-21T23:25:14.999 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-02-21T23:25:15.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.130+0000 7f84d1cad640 1 Processor -- start 2024-02-21T23:25:15.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.130+0000 7f84d1cad640 1 -- start start 2024-02-21T23:25:15.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.131+0000 7f84d1cad640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc152160 0x7f84cc0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:15.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.131+0000 7f84d1cad640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84cc1580e0 con 0x7f84cc0c9c30 2024-02-21T23:25:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.131+0000 7f84d1cad640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84cc1583c0 con 0x7f84cc0cdd40 2024-02-21T23:25:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.131+0000 7f84d1cad640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc1586a0 con 0x7f84cc152160 2024-02-21T23:25:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.132+0000 7f84cbfff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f84cc0cdd40 0x7f84cc15b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:45638/0 (socket says 172.21.15.133:45638) 2024-02-21T23:25:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.132+0000 7f84cbfff640 1 -- 172.21.15.133:0/2999426971 learned_addr learned my addr 172.21.15.133:0/2999426971 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:15.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.134+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003922852 0 0) 0x7f84cc1580e0 con 0x7f84cc0c9c30 2024-02-21T23:25:15.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.134+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b4003520 con 0x7f84cc0c9c30 2024-02-21T23:25:15.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.134+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17882806 0 0) 0x7f84cc1583c0 con 0x7f84cc0cdd40 2024-02-21T23:25:15.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.134+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84cc1580e0 con 0x7f84cc0cdd40 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3943308270 0 0) 0x7f84b4003520 con 0x7f84cc0c9c30 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc152160 msgr2=0x7f84cc0cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc152160 0x7f84cc0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 >> v1:172.21.15.142:6789/0 conn(0x7f84cc0cdd40 legacy=0x7f84cc15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc158980 con 0x7f84cc0c9c30 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 797941756 0 0) 0x7f84c0001670 con 0x7f84cc0c9c30 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 >> v1:172.21.15.133:6789/0 conn(0x7f84cc0c9c30 legacy=0x7f84cc0c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc152160 0x7f84b4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b4004120 0x7f84b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b4004e60 0x7f84b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc1586a0 con 0x7f84b4004120 2024-02-21T23:25:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4005570 con 0x7f84b4004e60 2024-02-21T23:25:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.138+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b400bf00 con 0x7f84cc152160 2024-02-21T23:25:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.140+0000 7f84caffd640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b4004120 0x7f84b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:15.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.142+0000 7f84cbfff640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b4004e60 0x7f84b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:15.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.143+0000 7f84caffd640 1 -- 172.21.15.133:0/2999426971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc152160 msgr2=0x7f84b4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:15.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.143+0000 7f84caffd640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc152160 0x7f84b4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:15.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.143+0000 7f84caffd640 1 -- 172.21.15.133:0/2999426971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b4004e60 msgr2=0x7f84b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:15.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.143+0000 7f84caffd640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b4004e60 0x7f84b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:15.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.143+0000 7f84caffd640 1 -- 172.21.15.133:0/2999426971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4003520 con 0x7f84b4004120 2024-02-21T23:25:15.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.145+0000 7f84caffd640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b4004120 0x7f84b4004800 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007ee0 tx=0x7f84b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8a5f24d8afcfa6 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:15.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.148+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0039070 con 0x7f84b4004120 2024-02-21T23:25:15.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.148+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c002c8b0 con 0x7f84b4004120 2024-02-21T23:25:15.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.148+0000 7f84ca7fc640 1 -- 172.21.15.133:0/2999426971 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0007b30 con 0x7f84b4004120 2024-02-21T23:25:15.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.149+0000 7f84d1cad640 1 -- 172.21.15.133:0/2999426971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b4004120 msgr2=0x7f84b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:15.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.149+0000 7f84d1cad640 1 --2- 172.21.15.133:0/2999426971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b4004120 0x7f84b4004800 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007ee0 tx=0x7f84b4005830 comp rx=0 tx=0).stop 2024-02-21T23:25:15.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.149+0000 7f84cbfff640 1 -- 172.21.15.133:0/2999426971 reap_dead start 2024-02-21T23:25:15.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.149+0000 7f84d1cad640 1 -- 172.21.15.133:0/2999426971 shutdown_connections 2024-02-21T23:25:15.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.149+0000 7f84d1cad640 1 -- 172.21.15.133:0/2999426971 >> 172.21.15.133:0/2999426971 conn(0x7f84cc06f380 msgr2=0x7f84cc06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:15.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.149+0000 7f84d1cad640 1 -- 172.21.15.133:0/2999426971 shutdown_connections 2024-02-21T23:25:15.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.149+0000 7f84d1cad640 1 -- 172.21.15.133:0/2999426971 wait complete. 2024-02-21T23:25:15.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.150+0000 7f84d1cad640 1 Processor -- start 2024-02-21T23:25:15.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.151+0000 7f84d1cad640 1 -- start start 2024-02-21T23:25:15.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.151+0000 7f84d1cad640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc0cab30 0x7f84cc0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:15.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.151+0000 7f84d1cad640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc0caf10 0x7f84cc14c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:15.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.151+0000 7f84d1cad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84cc157ba0 0x7f84cc14c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:15.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.151+0000 7f84d1cad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc0c9790 con 0x7f84cc157ba0 2024-02-21T23:25:15.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.151+0000 7f84d1cad640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc152540 con 0x7f84cc0cab30 2024-02-21T23:25:15.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.151+0000 7f84d1cad640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc0c9910 con 0x7f84cc0caf10 2024-02-21T23:25:15.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.152+0000 7f84cbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84cc157ba0 0x7f84cc14c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:15.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.152+0000 7f84cb7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc0cab30 0x7f84cc0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:15.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.152+0000 7f84cbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84cc157ba0 0x7f84cc14c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51246/0 (socket says 172.21.15.133:51246) 2024-02-21T23:25:15.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.152+0000 7f84cbfff640 1 -- 172.21.15.133:0/1574626140 learned_addr learned my addr 172.21.15.133:0/1574626140 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:15.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.155+0000 7f84cbfff640 1 -- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc0caf10 msgr2=0x7f84cc14c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:15.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.155+0000 7f84cbfff640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc0caf10 0x7f84cc14c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:15.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.155+0000 7f84cbfff640 1 -- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc0cab30 msgr2=0x7f84cc0c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:15.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.155+0000 7f84cbfff640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc0cab30 0x7f84cc0c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:15.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.155+0000 7f84cbfff640 1 -- 172.21.15.133:0/1574626140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc0c6d80 con 0x7f84cc157ba0 2024-02-21T23:25:15.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.158+0000 7f84cbfff640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84cc157ba0 0x7f84cc14c920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84cc06ff70 tx=0x7f84c4001870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:15.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.161+0000 7f84b3fff640 1 -- 172.21.15.133:0/1574626140 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4012560 con 0x7f84cc157ba0 2024-02-21T23:25:15.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.161+0000 7f84b3fff640 1 -- 172.21.15.133:0/1574626140 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c4018070 con 0x7f84cc157ba0 2024-02-21T23:25:15.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.161+0000 7f84b3fff640 1 -- 172.21.15.133:0/1574626140 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4012960 con 0x7f84cc157ba0 2024-02-21T23:25:15.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.161+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84cc0c77c0 con 0x7f84cc157ba0 2024-02-21T23:25:15.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.161+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc06c070 con 0x7f84cc157ba0 2024-02-21T23:25:15.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.162+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84cc06a130 con 0x7f84cc157ba0 2024-02-21T23:25:15.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.406+0000 7f84b3fff640 1 -- 172.21.15.133:0/1574626140 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84c4012b00 con 0x7f84cc157ba0 2024-02-21T23:25:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.406+0000 7f84b3fff640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f84b806b550 0x7f84b806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.406+0000 7f84cb7fe640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f84b806b550 0x7f84b806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.407+0000 7f84cb7fe640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f84b806b550 0x7f84b806da10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f84bc004b90 tx=0x7f84bc00d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:15.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.421+0000 7f84b3fff640 1 -- 172.21.15.133:0/1574626140 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f84c4012df0 con 0x7f84cc157ba0 2024-02-21T23:25:15.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.421+0000 7f84b3fff640 1 -- 172.21.15.133:0/1574626140 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f84cc0c6d80 con 0x7f84cc157ba0 2024-02-21T23:25:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:15.596+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f84cc1efc60 con 0x7f84cc157ba0 2024-02-21T23:25:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.091+0000 7f84b3fff640 1 -- 172.21.15.133:0/1574626140 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f84c4059740 con 0x7f84cc157ba0 2024-02-21T23:25:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-02-21T23:25:16.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f84b806b550 msgr2=0x7f84b806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:16.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f84b806b550 0x7f84b806da10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f84bc004b90 tx=0x7f84bc00d040 comp rx=0 tx=0).stop 2024-02-21T23:25:16.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84cc157ba0 msgr2=0x7f84cc14c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:16.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84cc157ba0 0x7f84cc14c920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84cc06ff70 tx=0x7f84c4001870 comp rx=0 tx=0).stop 2024-02-21T23:25:16.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 shutdown_connections 2024-02-21T23:25:16.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f84b806b550 0x7f84b806da10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84cc157ba0 0x7f84cc14c920 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84cc0caf10 0x7f84cc14c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 --2- 172.21.15.133:0/1574626140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc0cab30 0x7f84cc0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 >> 172.21.15.133:0/1574626140 conn(0x7f84cc06f380 msgr2=0x7f84cc06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:16.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 shutdown_connections 2024-02-21T23:25:16.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.093+0000 7f84d1cad640 1 -- 172.21.15.133:0/1574626140 wait complete. 2024-02-21T23:25:16.091 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-21T23:25:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.217+0000 7f12ede36640 1 Processor -- start 2024-02-21T23:25:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.217+0000 7f12ede36640 1 -- start start 2024-02-21T23:25:16.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.218+0000 7f12ede36640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f12e8154240 0x7f12e80ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:16.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.218+0000 7f12ede36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e80703c0 0x7f12e80c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:16.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.218+0000 7f12ede36640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f12e8070800 0x7f12e80c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:16.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.218+0000 7f12ede36640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e80cd300 con 0x7f12e8154240 2024-02-21T23:25:16.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.218+0000 7f12ede36640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e80cd570 con 0x7f12e8070800 2024-02-21T23:25:16.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.218+0000 7f12ede36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e815a130 con 0x7f12e80703c0 2024-02-21T23:25:16.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.219+0000 7f12e6ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e80703c0 0x7f12e80c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:16.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.219+0000 7f12e6ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e80703c0 0x7f12e80c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:33524/0 (socket says 172.21.15.133:33524) 2024-02-21T23:25:16.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.219+0000 7f12e6ffd640 1 -- 172.21.15.133:0/3954849702 learned_addr learned my addr 172.21.15.133:0/3954849702 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:16.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.221+0000 7f12e6ffd640 1 -- 172.21.15.133:0/3954849702 >> v2:172.21.15.142:3300/0 conn(0x7f12e8070800 msgr2=0x7f12e80c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:16.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.221+0000 7f12e6ffd640 1 --2- 172.21.15.133:0/3954849702 >> v2:172.21.15.142:3300/0 conn(0x7f12e8070800 0x7f12e80c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.221+0000 7f12e6ffd640 1 -- 172.21.15.133:0/3954849702 >> v2:172.21.15.133:3300/0 conn(0x7f12e8154240 msgr2=0x7f12e80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:16.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.221+0000 7f12e6ffd640 1 --2- 172.21.15.133:0/3954849702 >> v2:172.21.15.133:3300/0 conn(0x7f12e8154240 0x7f12e80ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.222+0000 7f12e6ffd640 1 -- 172.21.15.133:0/3954849702 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e815a410 con 0x7f12e80703c0 2024-02-21T23:25:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.224+0000 7f12e6ffd640 1 --2- 172.21.15.133:0/3954849702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e80703c0 0x7f12e80c90b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f12e806f410 tx=0x7f12dc02b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e79deda6c57b67 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.227+0000 7f12e67fc640 1 -- 172.21.15.133:0/3954849702 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc035020 con 0x7f12e80703c0 2024-02-21T23:25:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.227+0000 7f12e67fc640 1 -- 172.21.15.133:0/3954849702 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc031870 con 0x7f12e80703c0 2024-02-21T23:25:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.227+0000 7f12e67fc640 1 -- 172.21.15.133:0/3954849702 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc031c40 con 0x7f12e80703c0 2024-02-21T23:25:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.228+0000 7f12ede36640 1 -- 172.21.15.133:0/3954849702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e80703c0 msgr2=0x7f12e80c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.228+0000 7f12ede36640 1 --2- 172.21.15.133:0/3954849702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e80703c0 0x7f12e80c90b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f12e806f410 tx=0x7f12dc02b910 comp rx=0 tx=0).stop 2024-02-21T23:25:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.228+0000 7f12ede36640 1 -- 172.21.15.133:0/3954849702 shutdown_connections 2024-02-21T23:25:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.228+0000 7f12ede36640 1 --2- 172.21.15.133:0/3954849702 >> v2:172.21.15.142:3300/0 conn(0x7f12e8070800 0x7f12e80c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.228+0000 7f12ede36640 1 --2- 172.21.15.133:0/3954849702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e80703c0 0x7f12e80c90b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.228+0000 7f12ede36640 1 --2- 172.21.15.133:0/3954849702 >> v2:172.21.15.133:3300/0 conn(0x7f12e8154240 0x7f12e80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.228+0000 7f12ede36640 1 -- 172.21.15.133:0/3954849702 >> 172.21.15.133:0/3954849702 conn(0x7f12e8061b50 msgr2=0x7f12e8072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:16.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.228+0000 7f12ede36640 1 -- 172.21.15.133:0/3954849702 shutdown_connections 2024-02-21T23:25:16.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.228+0000 7f12ede36640 1 -- 172.21.15.133:0/3954849702 wait complete. 2024-02-21T23:25:16.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.229+0000 7f12ede36640 1 Processor -- start 2024-02-21T23:25:16.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.229+0000 7f12ede36640 1 -- start start 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.230+0000 7f12ede36640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80703c0 0x7f12e8155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.230+0000 7f12ede36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12e8070800 0x7f12e81559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.230+0000 7f12ede36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e8154240 0x7f12e80c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.230+0000 7f12ede36640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e8154620 con 0x7f12e8070800 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.230+0000 7f12ede36640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e815a9d0 con 0x7f12e80703c0 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.230+0000 7f12ede36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e80cc8a0 con 0x7f12e8154240 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.231+0000 7f12e7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e8154240 0x7f12e80c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.231+0000 7f12e77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80703c0 0x7f12e8155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.231+0000 7f12e7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e8154240 0x7f12e80c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:33534/0 (socket says 172.21.15.133:33534) 2024-02-21T23:25:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.231+0000 7f12e7fff640 1 -- 172.21.15.133:0/3697266438 learned_addr learned my addr 172.21.15.133:0/3697266438 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.234+0000 7f12e7fff640 1 -- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80703c0 msgr2=0x7f12e8155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.234+0000 7f12e7fff640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80703c0 0x7f12e8155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.234+0000 7f12e7fff640 1 -- 172.21.15.133:0/3697266438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12e8070800 msgr2=0x7f12e81559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.234+0000 7f12e7fff640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12e8070800 0x7f12e81559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.234+0000 7f12e7fff640 1 -- 172.21.15.133:0/3697266438 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e80cc2f0 con 0x7f12e8154240 2024-02-21T23:25:16.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.238+0000 7f12e7fff640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e8154240 0x7f12e80c5e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f12e000b9a0 tx=0x7f12e000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.241+0000 7f12cffff640 1 -- 172.21.15.133:0/3697266438 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0019070 con 0x7f12e8154240 2024-02-21T23:25:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.241+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e81f0970 con 0x7f12e8154240 2024-02-21T23:25:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.241+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e81f0c30 con 0x7f12e8154240 2024-02-21T23:25:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.242+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e80c9f40 con 0x7f12e8154240 2024-02-21T23:25:16.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.500+0000 7f12cffff640 1 -- 172.21.15.133:0/3697266438 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e000caa0 con 0x7f12e8154240 2024-02-21T23:25:16.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.500+0000 7f12cffff640 1 -- 172.21.15.133:0/3697266438 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0023060 con 0x7f12e8154240 2024-02-21T23:25:16.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.500+0000 7f12cffff640 1 -- 172.21.15.133:0/3697266438 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12e001c070 con 0x7f12e8154240 2024-02-21T23:25:16.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.500+0000 7f12cffff640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f12d406b4d0 0x7f12d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:16.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.501+0000 7f12e77fe640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f12d406b4d0 0x7f12d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:16.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.501+0000 7f12e77fe640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f12d406b4d0 0x7f12d406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009990 tx=0x7f12d8009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:16.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.503+0000 7f12cffff640 1 -- 172.21.15.133:0/3697266438 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f12e0056fa0 con 0x7f12e8154240 2024-02-21T23:25:16.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.515+0000 7f12cffff640 1 -- 172.21.15.133:0/3697266438 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f12e80c9f40 con 0x7f12e8154240 2024-02-21T23:25:16.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:16.726+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f12e80c4640 con 0x7f12e8154240 2024-02-21T23:25:17.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.404+0000 7f12cffff640 1 -- 172.21.15.133:0/3697266438 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f12e80c4640 con 0x7f12e8154240 2024-02-21T23:25:17.390 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 2 and data pool 3 2024-02-21T23:25:17.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f12d406b4d0 msgr2=0x7f12d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:17.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f12d406b4d0 0x7f12d406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009990 tx=0x7f12d8009110 comp rx=0 tx=0).stop 2024-02-21T23:25:17.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e8154240 msgr2=0x7f12e80c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:17.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e8154240 0x7f12e80c5e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f12e000b9a0 tx=0x7f12e000bda0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 shutdown_connections 2024-02-21T23:25:17.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f12d406b4d0 0x7f12d406d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12e8154240 0x7f12e80c5e20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12e8070800 0x7f12e81559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 --2- 172.21.15.133:0/3697266438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e80703c0 0x7f12e8155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 >> 172.21.15.133:0/3697266438 conn(0x7f12e8061b50 msgr2=0x7f12e806aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:17.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 shutdown_connections 2024-02-21T23:25:17.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.406+0000 7f12ede36640 1 -- 172.21.15.133:0/3697266438 wait complete. 2024-02-21T23:25:17.403 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-02-21T23:25:17.404 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-02-21T23:25:17.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.525+0000 7f51307e0640 1 Processor -- start 2024-02-21T23:25:17.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.525+0000 7f51307e0640 1 -- start start 2024-02-21T23:25:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.526+0000 7f51307e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c6f50 0x7f51280cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.526+0000 7f51307e0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51280c7c10 0x7f51280c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.526+0000 7f51307e0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51280c85e0 con 0x7f5128154600 2024-02-21T23:25:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.526+0000 7f51307e0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51280c74a0 con 0x7f51280c7c10 2024-02-21T23:25:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.526+0000 7f51307e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51280c7780 con 0x7f51280c6f50 2024-02-21T23:25:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.527+0000 7f512dd7b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c6f50 0x7f51280cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.528+0000 7f512dd7b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c6f50 0x7f51280cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:33548/0 (socket says 172.21.15.133:33548) 2024-02-21T23:25:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.528+0000 7f512dd7b640 1 -- 172.21.15.133:0/477503623 learned_addr learned my addr 172.21.15.133:0/477503623 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.530+0000 7f512dd7b640 1 -- 172.21.15.133:0/477503623 >> v2:172.21.15.133:3300/0 conn(0x7f51280c7c10 msgr2=0x7f51280c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.530+0000 7f512dd7b640 1 --2- 172.21.15.133:0/477503623 >> v2:172.21.15.133:3300/0 conn(0x7f51280c7c10 0x7f51280c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.530+0000 7f512dd7b640 1 -- 172.21.15.133:0/477503623 >> v1:172.21.15.133:6789/0 conn(0x7f5128154600 legacy=0x7f51280ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.530+0000 7f512dd7b640 1 -- 172.21.15.133:0/477503623 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512815ab40 con 0x7f51280c6f50 2024-02-21T23:25:17.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.532+0000 7f512dd7b640 1 --2- 172.21.15.133:0/477503623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c6f50 0x7f51280cbc00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5118009870 tx=0x7f511802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f49a65d336c076ad server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:17.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.536+0000 7f512d57a640 1 -- 172.21.15.133:0/477503623 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118034070 con 0x7f51280c6f50 2024-02-21T23:25:17.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.536+0000 7f512d57a640 1 -- 172.21.15.133:0/477503623 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5118030840 con 0x7f51280c6f50 2024-02-21T23:25:17.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.536+0000 7f512d57a640 1 -- 172.21.15.133:0/477503623 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118030c10 con 0x7f51280c6f50 2024-02-21T23:25:17.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.537+0000 7f51307e0640 1 -- 172.21.15.133:0/477503623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c6f50 msgr2=0x7f51280cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:17.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.537+0000 7f51307e0640 1 --2- 172.21.15.133:0/477503623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c6f50 0x7f51280cbc00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5118009870 tx=0x7f511802b980 comp rx=0 tx=0).stop 2024-02-21T23:25:17.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.537+0000 7f51307e0640 1 -- 172.21.15.133:0/477503623 shutdown_connections 2024-02-21T23:25:17.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.537+0000 7f51307e0640 1 --2- 172.21.15.133:0/477503623 >> v2:172.21.15.133:3300/0 conn(0x7f51280c7c10 0x7f51280c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.537+0000 7f51307e0640 1 --2- 172.21.15.133:0/477503623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c6f50 0x7f51280cbc00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.537+0000 7f51307e0640 1 -- 172.21.15.133:0/477503623 >> 172.21.15.133:0/477503623 conn(0x7f5128062000 msgr2=0x7f51280cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.537+0000 7f51307e0640 1 -- 172.21.15.133:0/477503623 shutdown_connections 2024-02-21T23:25:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.537+0000 7f51307e0640 1 -- 172.21.15.133:0/477503623 wait complete. 2024-02-21T23:25:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.538+0000 7f51307e0640 1 Processor -- start 2024-02-21T23:25:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.538+0000 7f51307e0640 1 -- start start 2024-02-21T23:25:17.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.539+0000 7f51307e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51280c6f50 0x7f51281e3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:17.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.539+0000 7f51307e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c7c10 0x7f51281e4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:17.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.539+0000 7f51307e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5128154600 0x7f51281e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:17.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.539+0000 7f51307e0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51280cb860 con 0x7f51280c6f50 2024-02-21T23:25:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.539+0000 7f51307e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51280cb580 con 0x7f5128154600 2024-02-21T23:25:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.539+0000 7f51307e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51280b5720 con 0x7f51280c7c10 2024-02-21T23:25:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.539+0000 7f512e57c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51280c6f50 0x7f51281e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.540+0000 7f512dd7b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c7c10 0x7f51281e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.540+0000 7f512e57c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51280c6f50 0x7f51281e3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51284/0 (socket says 172.21.15.133:51284) 2024-02-21T23:25:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.540+0000 7f512e57c640 1 -- 172.21.15.133:0/323422549 learned_addr learned my addr 172.21.15.133:0/323422549 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.543+0000 7f512e57c640 1 -- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c7c10 msgr2=0x7f51281e4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.543+0000 7f512e57c640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c7c10 0x7f51281e4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.543+0000 7f512e57c640 1 -- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5128154600 msgr2=0x7f51281e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.543+0000 7f512e57c640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5128154600 0x7f51281e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.543+0000 7f512e57c640 1 -- 172.21.15.133:0/323422549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280c8d70 con 0x7f51280c6f50 2024-02-21T23:25:17.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.545+0000 7f512e57c640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51280c6f50 0x7f51281e3600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f512400c430 tx=0x7f512400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:17.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.549+0000 7f511effd640 1 -- 172.21.15.133:0/323422549 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240052d0 con 0x7f51280c6f50 2024-02-21T23:25:17.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.549+0000 7f511effd640 1 -- 172.21.15.133:0/323422549 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5124005490 con 0x7f51280c6f50 2024-02-21T23:25:17.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.549+0000 7f511effd640 1 -- 172.21.15.133:0/323422549 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124007260 con 0x7f51280c6f50 2024-02-21T23:25:17.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.549+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51281f04d0 con 0x7f51280c6f50 2024-02-21T23:25:17.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.549+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51281f0a90 con 0x7f51280c6f50 2024-02-21T23:25:17.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.550+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512806a130 con 0x7f51280c6f50 2024-02-21T23:25:17.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.570+0000 7f511effd640 1 -- 172.21.15.133:0/323422549 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5124007400 con 0x7f51280c6f50 2024-02-21T23:25:17.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.570+0000 7f511effd640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f510c06b550 0x7f510c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:17.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.570+0000 7f512dd7b640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f510c06b550 0x7f510c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:17.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.571+0000 7f512dd7b640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f510c06b550 0x7f510c06da10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5118036000 tx=0x7f511803b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:17.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.808+0000 7f511effd640 1 -- 172.21.15.133:0/323422549 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5124056430 con 0x7f51280c6f50 2024-02-21T23:25:17.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.808+0000 7f511effd640 1 -- 172.21.15.133:0/323422549 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f5124058390 con 0x7f51280c6f50 2024-02-21T23:25:17.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:17.988+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f512814ddc0 con 0x7f51280c6f50 2024-02-21T23:25:18.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.138+0000 7f511effd640 1 -- 172.21.15.133:0/323422549 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f512405c430 con 0x7f51280c6f50 2024-02-21T23:25:18.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f510c06b550 msgr2=0x7f510c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:18.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f510c06b550 0x7f510c06da10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5118036000 tx=0x7f511803b040 comp rx=0 tx=0).stop 2024-02-21T23:25:18.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51280c6f50 msgr2=0x7f51281e3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:18.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51280c6f50 0x7f51281e3600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f512400c430 tx=0x7f512400c8f0 comp rx=0 tx=0).stop 2024-02-21T23:25:18.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 shutdown_connections 2024-02-21T23:25:18.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f510c06b550 0x7f510c06da10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:18.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5128154600 0x7f51281e4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:18.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51280c7c10 0x7f51281e4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:18.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 --2- 172.21.15.133:0/323422549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51280c6f50 0x7f51281e3600 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:18.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 >> 172.21.15.133:0/323422549 conn(0x7f5128062000 msgr2=0x7f5128152790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:18.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 shutdown_connections 2024-02-21T23:25:18.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.140+0000 7f51307e0640 1 -- 172.21.15.133:0/323422549 wait complete. 2024-02-21T23:25:18.138 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-02-21T23:25:18.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.265+0000 7fd594aa6640 1 Processor -- start 2024-02-21T23:25:18.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.265+0000 7fd594aa6640 1 -- start start 2024-02-21T23:25:18.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.266+0000 7fd594aa6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd59015ac80 0x7fd590165a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:18.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.266+0000 7fd594aa6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd590073590 con 0x7fd59015a190 2024-02-21T23:25:18.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.266+0000 7fd594aa6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5900c67b0 con 0x7fd590072f50 2024-02-21T23:25:18.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.266+0000 7fd594aa6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5900c5a60 con 0x7fd59015ac80 2024-02-21T23:25:18.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.266+0000 7fd58ed9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd59015ac80 0x7fd590165a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:18.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.267+0000 7fd58e59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd59015a190 0x7fd5900c7850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51970/0 (socket says 172.21.15.133:51970) 2024-02-21T23:25:18.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.267+0000 7fd58e59c640 1 -- 172.21.15.133:0/3250967584 learned_addr learned my addr 172.21.15.133:0/3250967584 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:18.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.271+0000 7fd58ed9d640 1 -- 172.21.15.133:0/3250967584 >> v1:172.21.15.142:6789/0 conn(0x7fd590072f50 legacy=0x7fd59015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:18.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.271+0000 7fd58ed9d640 1 -- 172.21.15.133:0/3250967584 >> v1:172.21.15.133:6789/0 conn(0x7fd59015a190 legacy=0x7fd5900c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:18.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.271+0000 7fd58ed9d640 1 -- 172.21.15.133:0/3250967584 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5900c6a90 con 0x7fd59015ac80 2024-02-21T23:25:18.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.277+0000 7fd58ed9d640 1 --2- 172.21.15.133:0/3250967584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd59015ac80 0x7fd590165a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd5880097a0 tx=0x7fd58802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d20ca769353048 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:18.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd58d59a640 1 -- 172.21.15.133:0/3250967584 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58802f810 con 0x7fd59015ac80 2024-02-21T23:25:18.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd58d59a640 1 -- 172.21.15.133:0/3250967584 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58802f9b0 con 0x7fd59015ac80 2024-02-21T23:25:18.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd58d59a640 1 -- 172.21.15.133:0/3250967584 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58802fd80 con 0x7fd59015ac80 2024-02-21T23:25:18.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd594aa6640 1 -- 172.21.15.133:0/3250967584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd59015ac80 msgr2=0x7fd590165a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:18.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd594aa6640 1 --2- 172.21.15.133:0/3250967584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd59015ac80 0x7fd590165a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd5880097a0 tx=0x7fd58802b9f0 comp rx=0 tx=0).stop 2024-02-21T23:25:18.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd594aa6640 1 -- 172.21.15.133:0/3250967584 shutdown_connections 2024-02-21T23:25:18.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd594aa6640 1 --2- 172.21.15.133:0/3250967584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd59015ac80 0x7fd590165a50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:18.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd594aa6640 1 -- 172.21.15.133:0/3250967584 >> 172.21.15.133:0/3250967584 conn(0x7fd5900c88a0 msgr2=0x7fd5900c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:18.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd594aa6640 1 -- 172.21.15.133:0/3250967584 shutdown_connections 2024-02-21T23:25:18.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.284+0000 7fd594aa6640 1 -- 172.21.15.133:0/3250967584 wait complete. 2024-02-21T23:25:18.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.286+0000 7fd594aa6640 1 Processor -- start 2024-02-21T23:25:18.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.286+0000 7fd594aa6640 1 -- start start 2024-02-21T23:25:18.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.287+0000 7fd594aa6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd590072f50 0x7fd5901e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:18.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.287+0000 7fd594aa6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59015a190 0x7fd5901f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:18.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.287+0000 7fd594aa6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59015ac80 0x7fd59015d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:18.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.287+0000 7fd594aa6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900c7330 con 0x7fd590072f50 2024-02-21T23:25:18.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.287+0000 7fd594aa6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900c7050 con 0x7fd59015ac80 2024-02-21T23:25:18.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.287+0000 7fd594aa6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59006aec0 con 0x7fd59015a190 2024-02-21T23:25:18.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.288+0000 7fd58ed9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59015ac80 0x7fd59015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:18.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.288+0000 7fd58dd9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59015a190 0x7fd5901f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:18.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.288+0000 7fd58ed9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59015ac80 0x7fd59015d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:44654/0 (socket says 172.21.15.133:44654) 2024-02-21T23:25:18.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.288+0000 7fd58ed9d640 1 -- 172.21.15.133:0/901618683 learned_addr learned my addr 172.21.15.133:0/901618683 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.291+0000 7fd58dd9b640 1 -- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59015ac80 msgr2=0x7fd59015d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.291+0000 7fd58dd9b640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59015ac80 0x7fd59015d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.291+0000 7fd58dd9b640 1 -- 172.21.15.133:0/901618683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd590072f50 msgr2=0x7fd5901e3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.291+0000 7fd58dd9b640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd590072f50 0x7fd5901e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.291+0000 7fd58dd9b640 1 -- 172.21.15.133:0/901618683 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5901e4490 con 0x7fd59015a190 2024-02-21T23:25:18.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.294+0000 7fd58dd9b640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59015a190 0x7fd5901f43b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c390 tx=0x7fd58400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.343+0000 7fd57effd640 1 -- 172.21.15.133:0/901618683 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584017070 con 0x7fd59015a190 2024-02-21T23:25:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.343+0000 7fd57effd640 1 -- 172.21.15.133:0/901618683 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5840052d0 con 0x7fd59015a190 2024-02-21T23:25:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.343+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5901f0980 con 0x7fd59015a190 2024-02-21T23:25:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.343+0000 7fd57effd640 1 -- 172.21.15.133:0/901618683 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584003830 con 0x7fd59015a190 2024-02-21T23:25:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.344+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5901f0f40 con 0x7fd59015a190 2024-02-21T23:25:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.344+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5900c3580 con 0x7fd59015a190 2024-02-21T23:25:18.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.600+0000 7fd57effd640 1 -- 172.21.15.133:0/901618683 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd58401a070 con 0x7fd59015a190 2024-02-21T23:25:18.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.601+0000 7fd57effd640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd57006b520 0x7fd57006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:18.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.601+0000 7fd58e59c640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd57006b520 0x7fd57006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:18.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.601+0000 7fd58e59c640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd57006b520 0x7fd57006d9e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd580002070 tx=0x7fd580001660 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.605+0000 7fd57effd640 1 -- 172.21.15.133:0/901618683 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7fd584056220 con 0x7fd59015a190 2024-02-21T23:25:18.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.615+0000 7fd57effd640 1 -- 172.21.15.133:0/901618683 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fd5900c3580 con 0x7fd59015a190 2024-02-21T23:25:18.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:18.789+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd5901595a0 con 0x7fd59015a190 2024-02-21T23:25:20.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.527+0000 7fd57effd640 1 -- 172.21.15.133:0/901618683 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fd5901595a0 con 0x7fd59015a190 2024-02-21T23:25:20.514 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data_ec' created 2024-02-21T23:25:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd57006b520 msgr2=0x7fd57006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd57006b520 0x7fd57006d9e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd580002070 tx=0x7fd580001660 comp rx=0 tx=0).stop 2024-02-21T23:25:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59015a190 msgr2=0x7fd5901f43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59015a190 0x7fd5901f43b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c390 tx=0x7fd58400c850 comp rx=0 tx=0).stop 2024-02-21T23:25:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 shutdown_connections 2024-02-21T23:25:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd57006b520 0x7fd57006d9e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59015ac80 0x7fd59015d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59015a190 0x7fd5901f43b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 --2- 172.21.15.133:0/901618683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd590072f50 0x7fd5901e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 >> 172.21.15.133:0/901618683 conn(0x7fd5900c88a0 msgr2=0x7fd590158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 shutdown_connections 2024-02-21T23:25:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.529+0000 7fd594aa6640 1 -- 172.21.15.133:0/901618683 wait complete. 2024-02-21T23:25:20.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-02-21T23:25:20.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.645+0000 7f9021695640 1 Processor -- start 2024-02-21T23:25:20.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.645+0000 7f9021695640 1 -- start start 2024-02-21T23:25:20.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.646+0000 7f9021695640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f901c154600 0x7f901c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:20.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.646+0000 7f9021695640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f901c0c7c10 0x7f901c0c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:20.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.646+0000 7f9021695640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c0c85a0 con 0x7f901c0c6f50 2024-02-21T23:25:20.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.646+0000 7f9021695640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c0c74a0 con 0x7f901c0c7c10 2024-02-21T23:25:20.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.646+0000 7f9021695640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c0c7780 con 0x7f901c154600 2024-02-21T23:25:20.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.647+0000 7f901affd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f901c154600 0x7f901c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:20.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.647+0000 7f9013fff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f901c0c6f50 0x7f901c0ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:45650/0 (socket says 172.21.15.133:45650) 2024-02-21T23:25:20.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.647+0000 7f9013fff640 1 -- 172.21.15.133:0/985300805 learned_addr learned my addr 172.21.15.133:0/985300805 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:20.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.647+0000 7f901b7fe640 1 --2- 172.21.15.133:0/985300805 >> v2:172.21.15.133:3300/0 conn(0x7f901c0c7c10 0x7f901c0c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:20.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.649+0000 7f901b7fe640 1 -- 172.21.15.133:0/985300805 >> v2:172.21.15.142:3300/0 conn(0x7f901c154600 msgr2=0x7f901c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:20.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.650+0000 7f901b7fe640 1 --2- 172.21.15.133:0/985300805 >> v2:172.21.15.142:3300/0 conn(0x7f901c154600 0x7f901c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.650+0000 7f901b7fe640 1 -- 172.21.15.133:0/985300805 >> v1:172.21.15.142:6789/0 conn(0x7f901c0c6f50 legacy=0x7f901c0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:20.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.650+0000 7f901b7fe640 1 -- 172.21.15.133:0/985300805 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c15ab40 con 0x7f901c0c7c10 2024-02-21T23:25:20.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.651+0000 7f901b7fe640 1 --2- 172.21.15.133:0/985300805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c0c7c10 0x7f901c0c8060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f901c06d270 tx=0x7f901402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff88a47ec9b3342 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.655+0000 7f901a7fc640 1 -- 172.21.15.133:0/985300805 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014039070 con 0x7f901c0c7c10 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.655+0000 7f901a7fc640 1 -- 172.21.15.133:0/985300805 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901402cc80 con 0x7f901c0c7c10 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.655+0000 7f901a7fc640 1 -- 172.21.15.133:0/985300805 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014034660 con 0x7f901c0c7c10 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.656+0000 7f9021695640 1 -- 172.21.15.133:0/985300805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c0c7c10 msgr2=0x7f901c0c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.656+0000 7f9021695640 1 --2- 172.21.15.133:0/985300805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c0c7c10 0x7f901c0c8060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f901c06d270 tx=0x7f901402bed0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.656+0000 7f9021695640 1 -- 172.21.15.133:0/985300805 shutdown_connections 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.656+0000 7f9021695640 1 --2- 172.21.15.133:0/985300805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c0c7c10 0x7f901c0c8060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.656+0000 7f9021695640 1 --2- 172.21.15.133:0/985300805 >> v2:172.21.15.142:3300/0 conn(0x7f901c154600 0x7f901c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.656+0000 7f9021695640 1 -- 172.21.15.133:0/985300805 >> 172.21.15.133:0/985300805 conn(0x7f901c062000 msgr2=0x7f901c0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:20.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.656+0000 7f9021695640 1 -- 172.21.15.133:0/985300805 shutdown_connections 2024-02-21T23:25:20.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.656+0000 7f9021695640 1 -- 172.21.15.133:0/985300805 wait complete. 2024-02-21T23:25:20.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.657+0000 7f9021695640 1 Processor -- start 2024-02-21T23:25:20.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.658+0000 7f9021695640 1 -- start start 2024-02-21T23:25:20.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.658+0000 7f9021695640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c0c6f50 0x7f901c14dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:20.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.658+0000 7f9021695640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f901c0c7c10 0x7f901c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:20.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.658+0000 7f9021695640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c154600 0x7f901c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:20.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.658+0000 7f9021695640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0cb860 con 0x7f901c154600 2024-02-21T23:25:20.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.658+0000 7f9021695640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0cb580 con 0x7f901c0c6f50 2024-02-21T23:25:20.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.658+0000 7f9021695640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0b5720 con 0x7f901c0c7c10 2024-02-21T23:25:20.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.659+0000 7f901b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c154600 0x7f901c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:20.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.659+0000 7f901affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c0c6f50 0x7f901c14dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:20.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.659+0000 7f901affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c0c6f50 0x7f901c14dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:44668/0 (socket says 172.21.15.133:44668) 2024-02-21T23:25:20.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.659+0000 7f901affd640 1 -- 172.21.15.133:0/3149352324 learned_addr learned my addr 172.21.15.133:0/3149352324 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:20.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.662+0000 7f901affd640 1 -- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f901c0c7c10 msgr2=0x7f901c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:20.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.662+0000 7f901affd640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f901c0c7c10 0x7f901c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.662+0000 7f901affd640 1 -- 172.21.15.133:0/3149352324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c154600 msgr2=0x7f901c155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:20.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.662+0000 7f901affd640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c154600 0x7f901c155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:20.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.662+0000 7f901affd640 1 -- 172.21.15.133:0/3149352324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c157940 con 0x7f901c0c6f50 2024-02-21T23:25:20.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.664+0000 7f901affd640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c0c6f50 0x7f901c14dee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f900c00e9e0 tx=0x7f900c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:20.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.667+0000 7f90137fe640 1 -- 172.21.15.133:0/3149352324 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c017070 con 0x7f901c0c6f50 2024-02-21T23:25:20.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.667+0000 7f90137fe640 1 -- 172.21.15.133:0/3149352324 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c00c9d0 con 0x7f901c0c6f50 2024-02-21T23:25:20.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.667+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c154c80 con 0x7f901c0c6f50 2024-02-21T23:25:20.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.667+0000 7f90137fe640 1 -- 172.21.15.133:0/3149352324 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c00cdc0 con 0x7f901c0c6f50 2024-02-21T23:25:20.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.667+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c155240 con 0x7f901c0c6f50 2024-02-21T23:25:20.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.669+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c06a130 con 0x7f901c0c6f50 2024-02-21T23:25:20.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.687+0000 7f90137fe640 1 -- 172.21.15.133:0/3149352324 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f900c01a070 con 0x7f901c0c6f50 2024-02-21T23:25:20.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.688+0000 7f90137fe640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ffc06b520 0x7f8ffc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:20.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.688+0000 7f9013fff640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ffc06b520 0x7f8ffc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:20.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.688+0000 7f9013fff640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ffc06b520 0x7f8ffc06d9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9004000f60 tx=0x7f9004007a80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:20.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.928+0000 7f90137fe640 1 -- 172.21.15.133:0/3149352324 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f900c057260 con 0x7f901c0c6f50 2024-02-21T23:25:20.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:20.939+0000 7f90137fe640 1 -- 172.21.15.133:0/3149352324 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f900c0591c0 con 0x7f901c0c6f50 2024-02-21T23:25:21.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.112+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f901c152f80 con 0x7f901c0c6f50 2024-02-21T23:25:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.660+0000 7f90137fe640 1 -- 172.21.15.133:0/3149352324 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f900c05d260 con 0x7f901c0c6f50 2024-02-21T23:25:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:set pool 4 allow_ec_overwrites to true 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ffc06b520 msgr2=0x7f8ffc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ffc06b520 0x7f8ffc06d9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9004000f60 tx=0x7f9004007a80 comp rx=0 tx=0).stop 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c0c6f50 msgr2=0x7f901c14dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c0c6f50 0x7f901c14dee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f900c00e9e0 tx=0x7f900c00eea0 comp rx=0 tx=0).stop 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 shutdown_connections 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ffc06b520 0x7f8ffc06d9e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c154600 0x7f901c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f901c0c7c10 0x7f901c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 --2- 172.21.15.133:0/3149352324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c0c6f50 0x7f901c14dee0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 >> 172.21.15.133:0/3149352324 conn(0x7f901c062000 msgr2=0x7f901c1599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 shutdown_connections 2024-02-21T23:25:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.662+0000 7f9021695640 1 -- 172.21.15.133:0/3149352324 wait complete. 2024-02-21T23:25:21.661 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-02-21T23:25:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.783+0000 7f47a141c640 1 Processor -- start 2024-02-21T23:25:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.783+0000 7f47a141c640 1 -- start start 2024-02-21T23:25:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.784+0000 7f47a141c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 0x7f479c0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.784+0000 7f47a141c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479c0717e0 con 0x7f479c0ce3c0 2024-02-21T23:25:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.784+0000 7f47a141c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479c158090 con 0x7f479c14e510 2024-02-21T23:25:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.784+0000 7f47a141c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0c84b0 con 0x7f479c0cdfe0 2024-02-21T23:25:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.785+0000 7f479b7fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f479c14e510 0x7f479c14e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:40314/0 (socket says 172.21.15.133:40314) 2024-02-21T23:25:21.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.785+0000 7f479b7fe640 1 -- 172.21.15.133:0/4024737088 learned_addr learned my addr 172.21.15.133:0/4024737088 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:21.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.785+0000 7f479affd640 1 --2- 172.21.15.133:0/4024737088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 0x7f479c0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:21.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.787+0000 7f4799ffb640 1 -- 172.21.15.133:0/4024737088 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886047391 0 0) 0x7f479c0717e0 con 0x7f479c0ce3c0 2024-02-21T23:25:21.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.787+0000 7f4799ffb640 1 -- 172.21.15.133:0/4024737088 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4784003520 con 0x7f479c0ce3c0 2024-02-21T23:25:21.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.787+0000 7f4799ffb640 1 -- 172.21.15.133:0/4024737088 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155018171 0 0) 0x7f479c158090 con 0x7f479c14e510 2024-02-21T23:25:21.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.788+0000 7f4799ffb640 1 -- 172.21.15.133:0/4024737088 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f479c0717e0 con 0x7f479c14e510 2024-02-21T23:25:21.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.788+0000 7f479affd640 1 -- 172.21.15.133:0/4024737088 >> v1:172.21.15.142:6789/0 conn(0x7f479c14e510 legacy=0x7f479c14e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:21.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.788+0000 7f479affd640 1 -- 172.21.15.133:0/4024737088 >> v1:172.21.15.133:6789/0 conn(0x7f479c0ce3c0 legacy=0x7f479c070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:21.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.788+0000 7f479affd640 1 -- 172.21.15.133:0/4024737088 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c158370 con 0x7f479c0cdfe0 2024-02-21T23:25:21.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.790+0000 7f479affd640 1 --2- 172.21.15.133:0/4024737088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 0x7f479c0c3250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47900097d0 tx=0x7f479002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49d4ed5e4d050980 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:21.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.794+0000 7f4799ffb640 1 -- 172.21.15.133:0/4024737088 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790035020 con 0x7f479c0cdfe0 2024-02-21T23:25:21.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.794+0000 7f4799ffb640 1 -- 172.21.15.133:0/4024737088 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4790031890 con 0x7f479c0cdfe0 2024-02-21T23:25:21.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.794+0000 7f4799ffb640 1 -- 172.21.15.133:0/4024737088 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790031c60 con 0x7f479c0cdfe0 2024-02-21T23:25:21.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.795+0000 7f47a141c640 1 -- 172.21.15.133:0/4024737088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 msgr2=0x7f479c0c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:21.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.795+0000 7f47a141c640 1 --2- 172.21.15.133:0/4024737088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 0x7f479c0c3250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47900097d0 tx=0x7f479002b930 comp rx=0 tx=0).stop 2024-02-21T23:25:21.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.795+0000 7f47a141c640 1 -- 172.21.15.133:0/4024737088 shutdown_connections 2024-02-21T23:25:21.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.795+0000 7f47a141c640 1 --2- 172.21.15.133:0/4024737088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 0x7f479c0c3250 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:21.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.795+0000 7f47a141c640 1 -- 172.21.15.133:0/4024737088 >> 172.21.15.133:0/4024737088 conn(0x7f479c06aec0 msgr2=0x7f479c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:21.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.795+0000 7f47a141c640 1 -- 172.21.15.133:0/4024737088 shutdown_connections 2024-02-21T23:25:21.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.795+0000 7f47a141c640 1 -- 172.21.15.133:0/4024737088 wait complete. 2024-02-21T23:25:21.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.796+0000 7f47a141c640 1 Processor -- start 2024-02-21T23:25:21.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.796+0000 7f47a141c640 1 -- start start 2024-02-21T23:25:21.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.797+0000 7f47a141c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 0x7f479c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:21.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.797+0000 7f47a141c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f479c0ce3c0 0x7f479c14a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:21.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.797+0000 7f47a141c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f479c14e510 0x7f479c1ed730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:21.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.797+0000 7f47a141c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c158930 con 0x7f479c0ce3c0 2024-02-21T23:25:21.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.797+0000 7f47a141c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c158ca0 con 0x7f479c14e510 2024-02-21T23:25:21.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.797+0000 7f47a141c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c071480 con 0x7f479c0cdfe0 2024-02-21T23:25:21.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.797+0000 7f479a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f479c0ce3c0 0x7f479c14a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:21.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.797+0000 7f479b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f479c14e510 0x7f479c1ed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:21.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.798+0000 7f479affd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 0x7f479c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:21.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.798+0000 7f479b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f479c14e510 0x7f479c1ed730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56656/0 (socket says 172.21.15.133:56656) 2024-02-21T23:25:21.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.798+0000 7f479b7fe640 1 -- 172.21.15.133:0/2375371415 learned_addr learned my addr 172.21.15.133:0/2375371415 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:21.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.801+0000 7f479a7fc640 1 -- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 msgr2=0x7f479c0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:21.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.801+0000 7f479a7fc640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 0x7f479c0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:21.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.801+0000 7f479a7fc640 1 -- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f479c14e510 msgr2=0x7f479c1ed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:21.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.801+0000 7f479a7fc640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f479c14e510 0x7f479c1ed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:21.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.801+0000 7f479a7fc640 1 -- 172.21.15.133:0/2375371415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c14abe0 con 0x7f479c0ce3c0 2024-02-21T23:25:21.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.801+0000 7f479b7fe640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f479c14e510 0x7f479c1ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:25:21.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.803+0000 7f479a7fc640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f479c0ce3c0 0x7f479c14a1c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f479c070c40 tx=0x7f478c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:21.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.807+0000 7f47837fe640 1 -- 172.21.15.133:0/2375371415 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c017070 con 0x7f479c0ce3c0 2024-02-21T23:25:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.807+0000 7f47837fe640 1 -- 172.21.15.133:0/2375371415 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c0052d0 con 0x7f479c0ce3c0 2024-02-21T23:25:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.807+0000 7f47837fe640 1 -- 172.21.15.133:0/2375371415 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c003830 con 0x7f479c0ce3c0 2024-02-21T23:25:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.807+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c1f0c40 con 0x7f479c0ce3c0 2024-02-21T23:25:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.807+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479c1f1200 con 0x7f479c0ce3c0 2024-02-21T23:25:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.808+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479c06a130 con 0x7f479c0ce3c0 2024-02-21T23:25:21.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.826+0000 7f47837fe640 1 -- 172.21.15.133:0/2375371415 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f478c01a070 con 0x7f479c0ce3c0 2024-02-21T23:25:21.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.826+0000 7f47837fe640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f478806b520 0x7f478806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:21.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.826+0000 7f479affd640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f478806b520 0x7f478806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:21.827+0000 7f479affd640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f478806b520 0x7f478806d9e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4790036000 tx=0x7f4790008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:22.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.061+0000 7f47837fe640 1 -- 172.21.15.133:0/2375371415 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f478c055fa0 con 0x7f479c0ce3c0 2024-02-21T23:25:22.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.062+0000 7f47837fe640 1 -- 172.21.15.133:0/2375371415 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f478c021080 con 0x7f479c0ce3c0 2024-02-21T23:25:22.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.278+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f479c15b960 con 0x7f479c0ce3c0 2024-02-21T23:25:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.616+0000 7f47837fe640 1 -- 172.21.15.133:0/2375371415 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f478c057f00 con 0x7f479c0ce3c0 2024-02-21T23:25:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:added data pool 4 to fsmap 2024-02-21T23:25:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f478806b520 msgr2=0x7f478806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f478806b520 0x7f478806d9e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4790036000 tx=0x7f4790008030 comp rx=0 tx=0).stop 2024-02-21T23:25:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f479c0ce3c0 msgr2=0x7f479c14a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f479c0ce3c0 0x7f479c14a1c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f479c070c40 tx=0x7f478c00c870 comp rx=0 tx=0).stop 2024-02-21T23:25:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 shutdown_connections 2024-02-21T23:25:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f478806b520 0x7f478806d9e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f479c14e510 0x7f479c1ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f479c0ce3c0 0x7f479c14a1c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 --2- 172.21.15.133:0/2375371415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479c0cdfe0 0x7f479c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 >> 172.21.15.133:0/2375371415 conn(0x7f479c06aec0 msgr2=0x7f479c155bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 shutdown_connections 2024-02-21T23:25:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.618+0000 7f47a141c640 1 -- 172.21.15.133:0/2375371415 wait complete. 2024-02-21T23:25:22.615 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:25:22.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.734+0000 7f2a79508640 1 Processor -- start 2024-02-21T23:25:22.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.734+0000 7f2a79508640 1 -- start start 2024-02-21T23:25:22.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.735+0000 7f2a79508640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a740c3dc0 0x7f2a740cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:22.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.735+0000 7f2a79508640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a740c41a0 0x7f2a74070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:22.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.735+0000 7f2a79508640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a74156e90 con 0x7f2a74150d30 2024-02-21T23:25:22.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.735+0000 7f2a79508640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a74157150 con 0x7f2a740c41a0 2024-02-21T23:25:22.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.735+0000 7f2a79508640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a74157430 con 0x7f2a740c3dc0 2024-02-21T23:25:22.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.736+0000 7f2a727fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a740c41a0 0x7f2a74070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:22.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.736+0000 7f2a727fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a740c41a0 0x7f2a74070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56664/0 (socket says 172.21.15.133:56664) 2024-02-21T23:25:22.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.736+0000 7f2a727fc640 1 -- 172.21.15.133:0/873859280 learned_addr learned my addr 172.21.15.133:0/873859280 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:22.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.738+0000 7f2a727fc640 1 -- 172.21.15.133:0/873859280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a740c3dc0 msgr2=0x7f2a740cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:22.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.738+0000 7f2a727fc640 1 --2- 172.21.15.133:0/873859280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a740c3dc0 0x7f2a740cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:22.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.738+0000 7f2a727fc640 1 -- 172.21.15.133:0/873859280 >> v1:172.21.15.133:6789/0 conn(0x7f2a74150d30 legacy=0x7f2a7415a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:22.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.738+0000 7f2a727fc640 1 -- 172.21.15.133:0/873859280 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a74157710 con 0x7f2a740c41a0 2024-02-21T23:25:22.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.742+0000 7f2a727fc640 1 --2- 172.21.15.133:0/873859280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a740c41a0 0x7f2a74070410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009870 tx=0x7f2a6402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e912356fb0a0cbe server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.792+0000 7f2a71ffb640 1 -- 172.21.15.133:0/873859280 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a64034070 con 0x7f2a740c41a0 2024-02-21T23:25:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.792+0000 7f2a71ffb640 1 -- 172.21.15.133:0/873859280 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a64030840 con 0x7f2a740c41a0 2024-02-21T23:25:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.792+0000 7f2a71ffb640 1 -- 172.21.15.133:0/873859280 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a64030c10 con 0x7f2a740c41a0 2024-02-21T23:25:22.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.793+0000 7f2a79508640 1 -- 172.21.15.133:0/873859280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a740c41a0 msgr2=0x7f2a74070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:22.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.793+0000 7f2a79508640 1 --2- 172.21.15.133:0/873859280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a740c41a0 0x7f2a74070410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009870 tx=0x7f2a6402b980 comp rx=0 tx=0).stop 2024-02-21T23:25:22.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.793+0000 7f2a79508640 1 -- 172.21.15.133:0/873859280 shutdown_connections 2024-02-21T23:25:22.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.793+0000 7f2a79508640 1 --2- 172.21.15.133:0/873859280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a740c41a0 0x7f2a74070410 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:22.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.793+0000 7f2a79508640 1 --2- 172.21.15.133:0/873859280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a740c3dc0 0x7f2a740cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:22.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.793+0000 7f2a79508640 1 -- 172.21.15.133:0/873859280 >> 172.21.15.133:0/873859280 conn(0x7f2a74061b50 msgr2=0x7f2a74072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:22.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.793+0000 7f2a79508640 1 -- 172.21.15.133:0/873859280 shutdown_connections 2024-02-21T23:25:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.793+0000 7f2a79508640 1 -- 172.21.15.133:0/873859280 wait complete. 2024-02-21T23:25:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.794+0000 7f2a79508640 1 Processor -- start 2024-02-21T23:25:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.794+0000 7f2a79508640 1 -- start start 2024-02-21T23:25:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.795+0000 7f2a79508640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a740c3dc0 0x7f2a741f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.795+0000 7f2a79508640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a740c41a0 0x7f2a741df310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.795+0000 7f2a79508640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a74150d30 0x7f2a74156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.795+0000 7f2a79508640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a74070190 con 0x7f2a740c41a0 2024-02-21T23:25:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.795+0000 7f2a79508640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a74157cd0 con 0x7f2a740c3dc0 2024-02-21T23:25:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.795+0000 7f2a79508640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a74070950 con 0x7f2a74150d30 2024-02-21T23:25:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.796+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a74150d30 0x7f2a74156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:22.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.796+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a74150d30 0x7f2a74156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:55960/0 (socket says 172.21.15.133:55960) 2024-02-21T23:25:22.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.796+0000 7f2a737fe640 1 -- 172.21.15.133:0/2501635667 learned_addr learned my addr 172.21.15.133:0/2501635667 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:22.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.797+0000 7f2a727fc640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a740c41a0 0x7f2a741df310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:22.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.797+0000 7f2a72ffd640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a740c3dc0 0x7f2a741f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:22.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.800+0000 7f2a737fe640 1 -- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a740c3dc0 msgr2=0x7f2a741f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:22.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.800+0000 7f2a737fe640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a740c3dc0 0x7f2a741f0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:22.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.800+0000 7f2a737fe640 1 -- 172.21.15.133:0/2501635667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a740c41a0 msgr2=0x7f2a741df310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:22.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.800+0000 7f2a737fe640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a740c41a0 0x7f2a741df310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:22.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.800+0000 7f2a737fe640 1 -- 172.21.15.133:0/2501635667 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a741df850 con 0x7f2a74150d30 2024-02-21T23:25:22.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.802+0000 7f2a737fe640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a74150d30 0x7f2a74156270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00c780 tx=0x7f2a6c00cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.805+0000 7f2a5b7fe640 1 -- 172.21.15.133:0/2501635667 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c017070 con 0x7f2a74150d30 2024-02-21T23:25:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.805+0000 7f2a5b7fe640 1 -- 172.21.15.133:0/2501635667 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c003da0 con 0x7f2a74150d30 2024-02-21T23:25:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.805+0000 7f2a5b7fe640 1 -- 172.21.15.133:0/2501635667 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0052b0 con 0x7f2a74150d30 2024-02-21T23:25:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.805+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a740cc9d0 con 0x7f2a74150d30 2024-02-21T23:25:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.805+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a740cbc50 con 0x7f2a74150d30 2024-02-21T23:25:22.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.806+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a740c6d70 con 0x7f2a74150d30 2024-02-21T23:25:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.824+0000 7f2a5b7fe640 1 -- 172.21.15.133:0/2501635667 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a6c01a070 con 0x7f2a74150d30 2024-02-21T23:25:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.825+0000 7f2a5b7fe640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2a6006b520 0x7f2a6006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.825+0000 7f2a72ffd640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2a6006b520 0x7f2a6006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.825+0000 7f2a72ffd640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2a6006b520 0x7f2a6006d9e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a680098e0 tx=0x7f2a68009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:22.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:22.827+0000 7f2a5b7fe640 1 -- 172.21.15.133:0/2501635667 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2a6c0564e0 con 0x7f2a74150d30 2024-02-21T23:25:23.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.069+0000 7f2a5b7fe640 1 -- 172.21.15.133:0/2501635667 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f2a740c6d70 con 0x7f2a74150d30 2024-02-21T23:25:23.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.282+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a7406a110 con 0x7f2a74150d30 2024-02-21T23:25:23.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.308+0000 7f2a5b7fe640 1 -- 172.21.15.133:0/2501635667 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6532 (secure 0 0 0) 0x7f2a6c021020 con 0x7f2a74150d30 2024-02-21T23:25:23.295 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:23.295 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1319340142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1319340142},{"type":"v1","addr":"172.21.15.133:6837","nonce":1319340142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1740931674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1740931674},{"type":"v1","addr":"172.21.15.142:6837","nonce":1740931674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3712864993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3712864993},{"type":"v1","addr":"172.21.15.142:6839","nonce":3712864993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2928905924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2928905924},{"type":"v1","addr":"172.21.15.133:6833","nonce":2928905924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3461010406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3461010406},{"type":"v1","addr":"172.21.15.133:6835","nonce":3461010406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-21T23:25:16.775942+0000","modified":"2024-02-21T23:25:22.297333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6835/1527124199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1527124199},{"type":"v1","addr":"172.21.15.142:6835","nonce":1527124199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-21T23:25:23.295 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-02-21T23:25:23.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.309+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2a6006b520 msgr2=0x7f2a6006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.309+0000 7f2a79508640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2a6006b520 0x7f2a6006d9e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a680098e0 tx=0x7f2a68009060 comp rx=0 tx=0).stop 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.309+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a74150d30 msgr2=0x7f2a74156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.309+0000 7f2a79508640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a74150d30 0x7f2a74156270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00c780 tx=0x7f2a6c00cc40 comp rx=0 tx=0).stop 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.310+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 shutdown_connections 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.310+0000 7f2a79508640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2a6006b520 0x7f2a6006d9e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.310+0000 7f2a79508640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a74150d30 0x7f2a74156270 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.310+0000 7f2a79508640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a740c41a0 0x7f2a741df310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.310+0000 7f2a79508640 1 --2- 172.21.15.133:0/2501635667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a740c3dc0 0x7f2a741f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.310+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 >> 172.21.15.133:0/2501635667 conn(0x7f2a74061b50 msgr2=0x7f2a740c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.310+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 shutdown_connections 2024-02-21T23:25:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.310+0000 7f2a79508640 1 -- 172.21.15.133:0/2501635667 wait complete. 2024-02-21T23:25:23.308 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:23.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.421+0000 7ffa34291640 1 Processor -- start 2024-02-21T23:25:23.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.421+0000 7ffa34291640 1 -- start start 2024-02-21T23:25:23.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.422+0000 7ffa34291640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c0c41a0 0x7ffa2c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.422+0000 7ffa34291640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa2c150d30 0x7ffa2c1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.422+0000 7ffa34291640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa2c156e90 con 0x7ffa2c0c3dc0 2024-02-21T23:25:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.422+0000 7ffa34291640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c157150 con 0x7ffa2c150d30 2024-02-21T23:25:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.422+0000 7ffa34291640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c157430 con 0x7ffa2c0c41a0 2024-02-21T23:25:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.423+0000 7ffa3182c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c0c41a0 0x7ffa2c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.423+0000 7ffa3282e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa2c150d30 0x7ffa2c1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.423+0000 7ffa3182c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c0c41a0 0x7ffa2c0cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:55976/0 (socket says 172.21.15.133:55976) 2024-02-21T23:25:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.423+0000 7ffa3182c640 1 -- 172.21.15.133:0/1858787655 learned_addr learned my addr 172.21.15.133:0/1858787655 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:23.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.425+0000 7ffa3182c640 1 -- 172.21.15.133:0/1858787655 >> v2:172.21.15.142:3300/0 conn(0x7ffa2c150d30 msgr2=0x7ffa2c1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:23.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.425+0000 7ffa3182c640 1 --2- 172.21.15.133:0/1858787655 >> v2:172.21.15.142:3300/0 conn(0x7ffa2c150d30 0x7ffa2c1511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.425+0000 7ffa3182c640 1 -- 172.21.15.133:0/1858787655 >> v1:172.21.15.142:6789/0 conn(0x7ffa2c0c3dc0 legacy=0x7ffa2c070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:23.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.425+0000 7ffa3182c640 1 -- 172.21.15.133:0/1858787655 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c157710 con 0x7ffa2c0c41a0 2024-02-21T23:25:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.427+0000 7ffa3182c640 1 --2- 172.21.15.133:0/1858787655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c0c41a0 0x7ffa2c0cd010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009870 tx=0x7ffa1c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22d3c03c6cc7b2d server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa3102b640 1 -- 172.21.15.133:0/1858787655 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c034070 con 0x7ffa2c0c41a0 2024-02-21T23:25:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa3102b640 1 -- 172.21.15.133:0/1858787655 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c030840 con 0x7ffa2c0c41a0 2024-02-21T23:25:23.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa3102b640 1 -- 172.21.15.133:0/1858787655 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c030c10 con 0x7ffa2c0c41a0 2024-02-21T23:25:23.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa34291640 1 -- 172.21.15.133:0/1858787655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c0c41a0 msgr2=0x7ffa2c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:23.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa34291640 1 --2- 172.21.15.133:0/1858787655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c0c41a0 0x7ffa2c0cd010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009870 tx=0x7ffa1c02b980 comp rx=0 tx=0).stop 2024-02-21T23:25:23.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa34291640 1 -- 172.21.15.133:0/1858787655 shutdown_connections 2024-02-21T23:25:23.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa34291640 1 --2- 172.21.15.133:0/1858787655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c0c41a0 0x7ffa2c0cd010 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa34291640 1 --2- 172.21.15.133:0/1858787655 >> v2:172.21.15.142:3300/0 conn(0x7ffa2c150d30 0x7ffa2c1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa34291640 1 -- 172.21.15.133:0/1858787655 >> 172.21.15.133:0/1858787655 conn(0x7ffa2c061b50 msgr2=0x7ffa2c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:23.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.431+0000 7ffa34291640 1 -- 172.21.15.133:0/1858787655 shutdown_connections 2024-02-21T23:25:23.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.432+0000 7ffa34291640 1 -- 172.21.15.133:0/1858787655 wait complete. 2024-02-21T23:25:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.433+0000 7ffa34291640 1 Processor -- start 2024-02-21T23:25:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.433+0000 7ffa34291640 1 -- start start 2024-02-21T23:25:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.433+0000 7ffa34291640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2c0c3dc0 0x7ffa2c1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.433+0000 7ffa34291640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa2c0c41a0 0x7ffa2c0c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.433+0000 7ffa34291640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c150d30 0x7ffa2c0ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.433+0000 7ffa34291640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c070190 con 0x7ffa2c0c41a0 2024-02-21T23:25:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.433+0000 7ffa34291640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c157cd0 con 0x7ffa2c0c3dc0 2024-02-21T23:25:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.433+0000 7ffa34291640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c15a600 con 0x7ffa2c150d30 2024-02-21T23:25:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.434+0000 7ffa3282e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c150d30 0x7ffa2c0ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.434+0000 7ffa3202d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2c0c3dc0 0x7ffa2c1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.434+0000 7ffa3182c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa2c0c41a0 0x7ffa2c0c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.434+0000 7ffa3202d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2c0c3dc0 0x7ffa2c1585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56688/0 (socket says 172.21.15.133:56688) 2024-02-21T23:25:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.434+0000 7ffa3202d640 1 -- 172.21.15.133:0/842612555 learned_addr learned my addr 172.21.15.133:0/842612555 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:23.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.437+0000 7ffa3282e640 1 -- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2c0c3dc0 msgr2=0x7ffa2c1585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:23.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.437+0000 7ffa3282e640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2c0c3dc0 0x7ffa2c1585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.437+0000 7ffa3282e640 1 -- 172.21.15.133:0/842612555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa2c0c41a0 msgr2=0x7ffa2c0c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:23.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.437+0000 7ffa3282e640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa2c0c41a0 0x7ffa2c0c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.437+0000 7ffa3282e640 1 -- 172.21.15.133:0/842612555 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c157ff0 con 0x7ffa2c150d30 2024-02-21T23:25:23.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.440+0000 7ffa3282e640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c150d30 0x7ffa2c0ca470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400e820 tx=0x7ffa2400ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.443+0000 7ffa22ffd640 1 -- 172.21.15.133:0/842612555 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24017070 con 0x7ffa2c150d30 2024-02-21T23:25:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.443+0000 7ffa22ffd640 1 -- 172.21.15.133:0/842612555 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2400c730 con 0x7ffa2c150d30 2024-02-21T23:25:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.443+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c1582b0 con 0x7ffa2c150d30 2024-02-21T23:25:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.443+0000 7ffa22ffd640 1 -- 172.21.15.133:0/842612555 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2400cb40 con 0x7ffa2c150d30 2024-02-21T23:25:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.443+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c1f3fa0 con 0x7ffa2c150d30 2024-02-21T23:25:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.444+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2c06a130 con 0x7ffa2c150d30 2024-02-21T23:25:23.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.463+0000 7ffa22ffd640 1 -- 172.21.15.133:0/842612555 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa2401a070 con 0x7ffa2c150d30 2024-02-21T23:25:23.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.464+0000 7ffa22ffd640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ffa1006b520 0x7ffa1006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:23.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.464+0000 7ffa3202d640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ffa1006b520 0x7ffa1006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:23.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.464+0000 7ffa3202d640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ffa1006b520 0x7ffa1006d9e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa28007fd0 tx=0x7ffa28009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:23.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.694+0000 7ffa22ffd640 1 -- 172.21.15.133:0/842612555 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffa240567d0 con 0x7ffa2c150d30 2024-02-21T23:25:23.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.705+0000 7ffa22ffd640 1 -- 172.21.15.133:0/842612555 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7ffa24058730 con 0x7ffa2c150d30 2024-02-21T23:25:23.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.884+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa2c0c9750 con 0x7ffa2c150d30 2024-02-21T23:25:23.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:23.999+0000 7ffa22ffd640 1 -- 172.21.15.133:0/842612555 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7ffa24081e20 con 0x7ffa2c150d30 2024-02-21T23:25:23.986 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:23.986 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:23.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ffa1006b520 msgr2=0x7ffa1006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:23.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ffa1006b520 0x7ffa1006d9e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa28007fd0 tx=0x7ffa28009040 comp rx=0 tx=0).stop 2024-02-21T23:25:23.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c150d30 msgr2=0x7ffa2c0ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:23.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c150d30 0x7ffa2c0ca470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400e820 tx=0x7ffa2400ece0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 shutdown_connections 2024-02-21T23:25:23.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ffa1006b520 0x7ffa1006d9e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2c150d30 0x7ffa2c0ca470 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa2c0c41a0 0x7ffa2c0c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 --2- 172.21.15.133:0/842612555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2c0c3dc0 0x7ffa2c1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:23.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 >> 172.21.15.133:0/842612555 conn(0x7ffa2c061b50 msgr2=0x7ffa2c155610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:23.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 shutdown_connections 2024-02-21T23:25:23.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.001+0000 7ffa34291640 1 -- 172.21.15.133:0/842612555 wait complete. 2024-02-21T23:25:23.999 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.113+0000 7fe7ed999640 1 Processor -- start 2024-02-21T23:25:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.113+0000 7fe7ed999640 1 -- start start 2024-02-21T23:25:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.114+0000 7fe7ed999640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e81540d0 0x7fe7e80cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.114+0000 7fe7ed999640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e80c9d20 con 0x7fe7e815aa30 2024-02-21T23:25:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.114+0000 7fe7ed999640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e80c96c0 con 0x7fe7e815e140 2024-02-21T23:25:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.114+0000 7fe7ed999640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e815d5a0 con 0x7fe7e81540d0 2024-02-21T23:25:24.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.115+0000 7fe7e77fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fe7e815e140 0x7fe7e8164930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:40332/0 (socket says 172.21.15.133:40332) 2024-02-21T23:25:24.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.115+0000 7fe7e77fe640 1 -- 172.21.15.133:0/2488053699 learned_addr learned my addr 172.21.15.133:0/2488053699 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:24.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.115+0000 7fe7e6ffd640 1 --2- 172.21.15.133:0/2488053699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e81540d0 0x7fe7e80cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.117+0000 7fe7e67fc640 1 -- 172.21.15.133:0/2488053699 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383338344 0 0) 0x7fe7e80c96c0 con 0x7fe7e815e140 2024-02-21T23:25:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.117+0000 7fe7e67fc640 1 -- 172.21.15.133:0/2488053699 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7d4003520 con 0x7fe7e815e140 2024-02-21T23:25:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.117+0000 7fe7e67fc640 1 -- 172.21.15.133:0/2488053699 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933448539 0 0) 0x7fe7e80c9d20 con 0x7fe7e815aa30 2024-02-21T23:25:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.117+0000 7fe7e67fc640 1 -- 172.21.15.133:0/2488053699 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e80c96c0 con 0x7fe7e815aa30 2024-02-21T23:25:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.118+0000 7fe7e6ffd640 1 -- 172.21.15.133:0/2488053699 >> v1:172.21.15.142:6789/0 conn(0x7fe7e815e140 legacy=0x7fe7e8164930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.118+0000 7fe7e6ffd640 1 -- 172.21.15.133:0/2488053699 >> v1:172.21.15.133:6789/0 conn(0x7fe7e815aa30 legacy=0x7fe7e815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.118+0000 7fe7e6ffd640 1 -- 172.21.15.133:0/2488053699 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e80c9fc0 con 0x7fe7e81540d0 2024-02-21T23:25:24.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.120+0000 7fe7e6ffd640 1 --2- 172.21.15.133:0/2488053699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e81540d0 0x7fe7e80cd900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097d0 tx=0x7fe7d802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22913442cbb7c97a server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:24.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.123+0000 7fe7e67fc640 1 -- 172.21.15.133:0/2488053699 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8035020 con 0x7fe7e81540d0 2024-02-21T23:25:24.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.123+0000 7fe7e67fc640 1 -- 172.21.15.133:0/2488053699 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d8031800 con 0x7fe7e81540d0 2024-02-21T23:25:24.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.123+0000 7fe7e67fc640 1 -- 172.21.15.133:0/2488053699 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8031bd0 con 0x7fe7e81540d0 2024-02-21T23:25:24.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.124+0000 7fe7ed999640 1 -- 172.21.15.133:0/2488053699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e81540d0 msgr2=0x7fe7e80cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.124+0000 7fe7ed999640 1 --2- 172.21.15.133:0/2488053699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e81540d0 0x7fe7e80cd900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097d0 tx=0x7fe7d802b8a0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.124+0000 7fe7ed999640 1 -- 172.21.15.133:0/2488053699 shutdown_connections 2024-02-21T23:25:24.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.124+0000 7fe7ed999640 1 --2- 172.21.15.133:0/2488053699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e81540d0 0x7fe7e80cd900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.124+0000 7fe7ed999640 1 -- 172.21.15.133:0/2488053699 >> 172.21.15.133:0/2488053699 conn(0x7fe7e806f7f0 msgr2=0x7fe7e8061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:24.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.124+0000 7fe7ed999640 1 -- 172.21.15.133:0/2488053699 shutdown_connections 2024-02-21T23:25:24.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.124+0000 7fe7ed999640 1 -- 172.21.15.133:0/2488053699 wait complete. 2024-02-21T23:25:24.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.125+0000 7fe7ed999640 1 Processor -- start 2024-02-21T23:25:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.125+0000 7fe7ed999640 1 -- start start 2024-02-21T23:25:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.126+0000 7fe7ed999640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e81540d0 0x7fe7e815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.126+0000 7fe7ed999640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e815aa30 0x7fe7e8154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.126+0000 7fe7ed999640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7e815e140 0x7fe7e8155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.126+0000 7fe7ed999640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80c8420 con 0x7fe7e815e140 2024-02-21T23:25:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.126+0000 7fe7ed999640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e806c040 con 0x7fe7e81540d0 2024-02-21T23:25:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.126+0000 7fe7ed999640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80c85a0 con 0x7fe7e815aa30 2024-02-21T23:25:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.127+0000 7fe7e6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e81540d0 0x7fe7e815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.127+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e815aa30 0x7fe7e8154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.127+0000 7fe7e77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7e815e140 0x7fe7e8155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.127+0000 7fe7e6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e81540d0 0x7fe7e815c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56694/0 (socket says 172.21.15.133:56694) 2024-02-21T23:25:24.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.127+0000 7fe7e6ffd640 1 -- 172.21.15.133:0/1153970499 learned_addr learned my addr 172.21.15.133:0/1153970499 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:24.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.130+0000 7fe7e77fe640 1 -- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e815aa30 msgr2=0x7fe7e8154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.130+0000 7fe7e77fe640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e815aa30 0x7fe7e8154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.130+0000 7fe7e77fe640 1 -- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e81540d0 msgr2=0x7fe7e815c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.130+0000 7fe7e77fe640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e81540d0 0x7fe7e815c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.130+0000 7fe7e77fe640 1 -- 172.21.15.133:0/1153970499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8157e00 con 0x7fe7e815e140 2024-02-21T23:25:24.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.132+0000 7fe7e77fe640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7e815e140 0x7fe7e8155190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0001580 tx=0x7fe7e000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:24.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.136+0000 7fe7e4ff9640 1 -- 172.21.15.133:0/1153970499 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0017070 con 0x7fe7e815e140 2024-02-21T23:25:24.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.136+0000 7fe7e4ff9640 1 -- 172.21.15.133:0/1153970499 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e000aba0 con 0x7fe7e815e140 2024-02-21T23:25:24.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.136+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e81580e0 con 0x7fe7e815e140 2024-02-21T23:25:24.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.136+0000 7fe7e4ff9640 1 -- 172.21.15.133:0/1153970499 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0012400 con 0x7fe7e815e140 2024-02-21T23:25:24.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.136+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e81586a0 con 0x7fe7e815e140 2024-02-21T23:25:24.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.137+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e8158a80 con 0x7fe7e815e140 2024-02-21T23:25:24.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.381+0000 7fe7e4ff9640 1 -- 172.21.15.133:0/1153970499 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7e001a070 con 0x7fe7e815e140 2024-02-21T23:25:24.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.382+0000 7fe7e4ff9640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe7c806b520 0x7fe7c806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.382+0000 7fe7e6ffd640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe7c806b520 0x7fe7c806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.383+0000 7fe7e6ffd640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe7c806b520 0x7fe7c806d9e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7d802b8e0 tx=0x7fe7d8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:24.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.395+0000 7fe7e4ff9640 1 -- 172.21.15.133:0/1153970499 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe7e001de30 con 0x7fe7e815e140 2024-02-21T23:25:24.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.396+0000 7fe7e4ff9640 1 -- 172.21.15.133:0/1153970499 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fe7e8158a80 con 0x7fe7e815e140 2024-02-21T23:25:24.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.571+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7e81545e0 con 0x7fe7e815e140 2024-02-21T23:25:24.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.617+0000 7fe7e4ff9640 1 -- 172.21.15.133:0/1153970499 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fe7e81545e0 con 0x7fe7e815e140 2024-02-21T23:25:24.604 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:24.605 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:24.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe7c806b520 msgr2=0x7fe7c806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe7c806b520 0x7fe7c806d9e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7d802b8e0 tx=0x7fe7d8008030 comp rx=0 tx=0).stop 2024-02-21T23:25:24.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7e815e140 msgr2=0x7fe7e8155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7e815e140 0x7fe7e8155190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0001580 tx=0x7fe7e000cee0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 shutdown_connections 2024-02-21T23:25:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe7c806b520 0x7fe7c806d9e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7e815e140 0x7fe7e8155190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7e815aa30 0x7fe7e8154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 --2- 172.21.15.133:0/1153970499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e81540d0 0x7fe7e815c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 >> 172.21.15.133:0/1153970499 conn(0x7fe7e806f7f0 msgr2=0x7fe7e8152de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 shutdown_connections 2024-02-21T23:25:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.619+0000 7fe7ed999640 1 -- 172.21.15.133:0/1153970499 wait complete. 2024-02-21T23:25:24.632 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:25:24.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.754+0000 7f43fd236640 1 Processor -- start 2024-02-21T23:25:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.755+0000 7f43fd236640 1 -- start start 2024-02-21T23:25:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.755+0000 7f43fd236640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f43f815aa30 0x7f43f815da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.755+0000 7f43fd236640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f43f80c8940 0x7f43f815df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.755+0000 7f43fd236640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f80c9d20 con 0x7f43f81540d0 2024-02-21T23:25:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.755+0000 7f43fd236640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f815d5a0 con 0x7f43f815aa30 2024-02-21T23:25:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.755+0000 7f43fd236640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f80c96c0 con 0x7f43f80c8940 2024-02-21T23:25:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.756+0000 7f43f759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f43f80c8940 0x7f43f815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.756+0000 7f43f659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f43f815aa30 0x7f43f815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.756+0000 7f43f759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f43f80c8940 0x7f43f815df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56706/0 (socket says 172.21.15.133:56706) 2024-02-21T23:25:24.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.756+0000 7f43f759e640 1 -- 172.21.15.133:0/3304184346 learned_addr learned my addr 172.21.15.133:0/3304184346 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.758+0000 7f43f759e640 1 -- 172.21.15.133:0/3304184346 >> v2:172.21.15.133:3300/0 conn(0x7f43f815aa30 msgr2=0x7f43f815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.758+0000 7f43f759e640 1 --2- 172.21.15.133:0/3304184346 >> v2:172.21.15.133:3300/0 conn(0x7f43f815aa30 0x7f43f815da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.758+0000 7f43f759e640 1 -- 172.21.15.133:0/3304184346 >> v1:172.21.15.133:6789/0 conn(0x7f43f81540d0 legacy=0x7f43f80cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.758+0000 7f43f759e640 1 -- 172.21.15.133:0/3304184346 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f80c9fc0 con 0x7f43f80c8940 2024-02-21T23:25:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.760+0000 7f43f759e640 1 --2- 172.21.15.133:0/3304184346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 0x7f43f815df70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43f815d720 tx=0x7f43e802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650382b87ee9e2b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:24.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.764+0000 7f43f5d9b640 1 -- 172.21.15.133:0/3304184346 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8035020 con 0x7f43f80c8940 2024-02-21T23:25:24.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.764+0000 7f43f5d9b640 1 -- 172.21.15.133:0/3304184346 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e8031870 con 0x7f43f80c8940 2024-02-21T23:25:24.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.764+0000 7f43f5d9b640 1 -- 172.21.15.133:0/3304184346 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8031c40 con 0x7f43f80c8940 2024-02-21T23:25:24.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.764+0000 7f43fd236640 1 -- 172.21.15.133:0/3304184346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 msgr2=0x7f43f815df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.764+0000 7f43fd236640 1 --2- 172.21.15.133:0/3304184346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 0x7f43f815df70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43f815d720 tx=0x7f43e802b910 comp rx=0 tx=0).stop 2024-02-21T23:25:24.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.765+0000 7f43fd236640 1 -- 172.21.15.133:0/3304184346 shutdown_connections 2024-02-21T23:25:24.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.765+0000 7f43fd236640 1 --2- 172.21.15.133:0/3304184346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 0x7f43f815df70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.765+0000 7f43fd236640 1 --2- 172.21.15.133:0/3304184346 >> v2:172.21.15.133:3300/0 conn(0x7f43f815aa30 0x7f43f815da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.765+0000 7f43fd236640 1 -- 172.21.15.133:0/3304184346 >> 172.21.15.133:0/3304184346 conn(0x7f43f806f7f0 msgr2=0x7f43f8061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:24.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.765+0000 7f43fd236640 1 -- 172.21.15.133:0/3304184346 shutdown_connections 2024-02-21T23:25:24.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.765+0000 7f43fd236640 1 -- 172.21.15.133:0/3304184346 wait complete. 2024-02-21T23:25:24.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.766+0000 7f43fd236640 1 Processor -- start 2024-02-21T23:25:24.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.766+0000 7f43fd236640 1 -- start start 2024-02-21T23:25:24.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.767+0000 7f43fd236640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 0x7f43f8155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.767+0000 7f43fd236640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43f81540d0 0x7f43f8152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.767+0000 7f43fd236640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815aa30 0x7f43f81efaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:24.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.767+0000 7f43fd236640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80ce010 con 0x7f43f81540d0 2024-02-21T23:25:24.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.767+0000 7f43fd236640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f806c040 con 0x7f43f80c8940 2024-02-21T23:25:24.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.767+0000 7f43fd236640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80ce190 con 0x7f43f815aa30 2024-02-21T23:25:24.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.767+0000 7f43f6d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 0x7f43f8155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.767+0000 7f43f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43f81540d0 0x7f43f8152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.767+0000 7f43f759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815aa30 0x7f43f81efaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:24.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.768+0000 7f43f6d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 0x7f43f8155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56718/0 (socket says 172.21.15.133:56718) 2024-02-21T23:25:24.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.768+0000 7f43f6d9d640 1 -- 172.21.15.133:0/84732418 learned_addr learned my addr 172.21.15.133:0/84732418 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:24.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.771+0000 7f43f659c640 1 -- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815aa30 msgr2=0x7f43f81efaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.771+0000 7f43f659c640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815aa30 0x7f43f81efaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.771+0000 7f43f659c640 1 -- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 msgr2=0x7f43f8155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:24.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.771+0000 7f43f659c640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 0x7f43f8155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:24.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.771+0000 7f43f659c640 1 -- 172.21.15.133:0/84732418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8153920 con 0x7f43f81540d0 2024-02-21T23:25:24.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.773+0000 7f43f659c640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43f81540d0 0x7f43f8152980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00e870 tx=0x7f43ec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:24.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.777+0000 7f43df7fe640 1 -- 172.21.15.133:0/84732418 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec017070 con 0x7f43f81540d0 2024-02-21T23:25:24.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.777+0000 7f43df7fe640 1 -- 172.21.15.133:0/84732418 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec00ee60 con 0x7f43f81540d0 2024-02-21T23:25:24.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.777+0000 7f43df7fe640 1 -- 172.21.15.133:0/84732418 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00cb60 con 0x7f43f81540d0 2024-02-21T23:25:24.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.777+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f8156cd0 con 0x7f43f81540d0 2024-02-21T23:25:24.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.777+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f8153640 con 0x7f43f81540d0 2024-02-21T23:25:24.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:24.778+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f806a130 con 0x7f43f81540d0 2024-02-21T23:25:25.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.022+0000 7f43df7fe640 1 -- 172.21.15.133:0/84732418 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43ec01a070 con 0x7f43f81540d0 2024-02-21T23:25:25.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.022+0000 7f43df7fe640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f43e406b4d0 0x7f43e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:25.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.022+0000 7f43f6d9d640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f43e406b4d0 0x7f43e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:25.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.023+0000 7f43f6d9d640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f43e406b4d0 0x7f43e406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f43f0007fd0 tx=0x7f43f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:25.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.035+0000 7f43df7fe640 1 -- 172.21.15.133:0/84732418 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f43ec057050 con 0x7f43f81540d0 2024-02-21T23:25:25.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.035+0000 7f43df7fe640 1 -- 172.21.15.133:0/84732418 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f43ec058fb0 con 0x7f43f81540d0 2024-02-21T23:25:25.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.252+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43f80c8f20 con 0x7f43f81540d0 2024-02-21T23:25:25.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.272+0000 7f43df7fe640 1 -- 172.21.15.133:0/84732418 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6532 (secure 0 0 0) 0x7f43ec05d050 con 0x7f43f81540d0 2024-02-21T23:25:25.259 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:25.259 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1319340142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1319340142},{"type":"v1","addr":"172.21.15.133:6837","nonce":1319340142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1740931674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1740931674},{"type":"v1","addr":"172.21.15.142:6837","nonce":1740931674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3712864993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3712864993},{"type":"v1","addr":"172.21.15.142:6839","nonce":3712864993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2928905924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2928905924},{"type":"v1","addr":"172.21.15.133:6833","nonce":2928905924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3461010406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3461010406},{"type":"v1","addr":"172.21.15.133:6835","nonce":3461010406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-21T23:25:16.775942+0000","modified":"2024-02-21T23:25:22.297333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6835/1527124199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1527124199},{"type":"v1","addr":"172.21.15.142:6835","nonce":1527124199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-21T23:25:25.260 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-02-21T23:25:25.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f43e406b4d0 msgr2=0x7f43e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:25.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f43e406b4d0 0x7f43e406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f43f0007fd0 tx=0x7f43f0009040 comp rx=0 tx=0).stop 2024-02-21T23:25:25.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43f81540d0 msgr2=0x7f43f8152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43f81540d0 0x7f43f8152980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00e870 tx=0x7f43ec00ed30 comp rx=0 tx=0).stop 2024-02-21T23:25:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 shutdown_connections 2024-02-21T23:25:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f43e406b4d0 0x7f43e406d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815aa30 0x7f43f81efaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43f81540d0 0x7f43f8152980 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 --2- 172.21.15.133:0/84732418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80c8940 0x7f43f8155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 >> 172.21.15.133:0/84732418 conn(0x7f43f806f7f0 msgr2=0x7f43f8061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 shutdown_connections 2024-02-21T23:25:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.274+0000 7f43fd236640 1 -- 172.21.15.133:0/84732418 wait complete. 2024-02-21T23:25:25.272 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:25.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.385+0000 7fa68f5cc640 1 Processor -- start 2024-02-21T23:25:25.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.385+0000 7fa68f5cc640 1 -- start start 2024-02-21T23:25:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.386+0000 7fa68f5cc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 0x7fa6880cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.386+0000 7fa68f5cc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa6880c41a0 0x7fa688070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.386+0000 7fa68f5cc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa688156e90 con 0x7fa688150d30 2024-02-21T23:25:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.386+0000 7fa68f5cc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa688157150 con 0x7fa6880c41a0 2024-02-21T23:25:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.386+0000 7fa68f5cc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa688157430 con 0x7fa6880c3dc0 2024-02-21T23:25:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.386+0000 7fa68cb67640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa6880c41a0 0x7fa688070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.386+0000 7fa68d368640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 0x7fa6880cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:25.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.387+0000 7fa68cb67640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa6880c41a0 0x7fa688070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56728/0 (socket says 172.21.15.133:56728) 2024-02-21T23:25:25.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.387+0000 7fa68cb67640 1 -- 172.21.15.133:0/938505019 learned_addr learned my addr 172.21.15.133:0/938505019 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:25.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.389+0000 7fa68cb67640 1 -- 172.21.15.133:0/938505019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 msgr2=0x7fa6880cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:25.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.389+0000 7fa68cb67640 1 --2- 172.21.15.133:0/938505019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 0x7fa6880cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.389+0000 7fa68cb67640 1 -- 172.21.15.133:0/938505019 >> v1:172.21.15.133:6789/0 conn(0x7fa688150d30 legacy=0x7fa68815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:25.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.389+0000 7fa68cb67640 1 -- 172.21.15.133:0/938505019 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688157710 con 0x7fa6880c41a0 2024-02-21T23:25:25.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.391+0000 7fa68cb67640 1 --2- 172.21.15.133:0/938505019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 0x7fa688070410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa678009870 tx=0x7fa67802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29da77a6e461b99 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa67ffff640 1 -- 172.21.15.133:0/938505019 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67802f840 con 0x7fa6880c41a0 2024-02-21T23:25:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa67ffff640 1 -- 172.21.15.133:0/938505019 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67802f9e0 con 0x7fa6880c41a0 2024-02-21T23:25:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa67ffff640 1 -- 172.21.15.133:0/938505019 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67802fdb0 con 0x7fa6880c41a0 2024-02-21T23:25:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa68f5cc640 1 -- 172.21.15.133:0/938505019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 msgr2=0x7fa688070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa68f5cc640 1 --2- 172.21.15.133:0/938505019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 0x7fa688070410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa678009870 tx=0x7fa67802ba20 comp rx=0 tx=0).stop 2024-02-21T23:25:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa68f5cc640 1 -- 172.21.15.133:0/938505019 shutdown_connections 2024-02-21T23:25:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa68f5cc640 1 --2- 172.21.15.133:0/938505019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 0x7fa688070410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa68f5cc640 1 --2- 172.21.15.133:0/938505019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 0x7fa6880cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa68f5cc640 1 -- 172.21.15.133:0/938505019 >> 172.21.15.133:0/938505019 conn(0x7fa688061b50 msgr2=0x7fa688072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa68f5cc640 1 -- 172.21.15.133:0/938505019 shutdown_connections 2024-02-21T23:25:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.395+0000 7fa68f5cc640 1 -- 172.21.15.133:0/938505019 wait complete. 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.397+0000 7fa68f5cc640 1 Processor -- start 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.397+0000 7fa68f5cc640 1 -- start start 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.397+0000 7fa68f5cc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 0x7fa6881f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.397+0000 7fa68f5cc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 0x7fa68814ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.397+0000 7fa68f5cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688150d30 0x7fa6881f1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.397+0000 7fa68f5cc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa688070190 con 0x7fa688150d30 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.397+0000 7fa68f5cc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa688157cd0 con 0x7fa6880c41a0 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.397+0000 7fa68f5cc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa688070950 con 0x7fa6880c3dc0 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.398+0000 7fa68db69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688150d30 0x7fa6881f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.398+0000 7fa68cb67640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 0x7fa68814ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.398+0000 7fa68d368640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 0x7fa6881f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.398+0000 7fa68cb67640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 0x7fa68814ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56734/0 (socket says 172.21.15.133:56734) 2024-02-21T23:25:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.398+0000 7fa68cb67640 1 -- 172.21.15.133:0/4171300192 learned_addr learned my addr 172.21.15.133:0/4171300192 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.401+0000 7fa68db69640 1 -- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 msgr2=0x7fa6881f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.401+0000 7fa68db69640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 0x7fa6881f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.401+0000 7fa68db69640 1 -- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 msgr2=0x7fa68814ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.401+0000 7fa68db69640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 0x7fa68814ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.401+0000 7fa68db69640 1 -- 172.21.15.133:0/4171300192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6881601f0 con 0x7fa688150d30 2024-02-21T23:25:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.401+0000 7fa68cb67640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 0x7fa68814ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:25:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.404+0000 7fa68db69640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688150d30 0x7fa6881f1460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c690 tx=0x7fa68400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.407+0000 7fa67dffb640 1 -- 172.21.15.133:0/4171300192 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684003b70 con 0x7fa688150d30 2024-02-21T23:25:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.407+0000 7fa67dffb640 1 -- 172.21.15.133:0/4171300192 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa684003d10 con 0x7fa688150d30 2024-02-21T23:25:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.407+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6880c3730 con 0x7fa688150d30 2024-02-21T23:25:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.407+0000 7fa67dffb640 1 -- 172.21.15.133:0/4171300192 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684005310 con 0x7fa688150d30 2024-02-21T23:25:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.407+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6881e97f0 con 0x7fa688150d30 2024-02-21T23:25:25.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.409+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68806a130 con 0x7fa688150d30 2024-02-21T23:25:25.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.426+0000 7fa67dffb640 1 -- 172.21.15.133:0/4171300192 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6840054b0 con 0x7fa688150d30 2024-02-21T23:25:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.426+0000 7fa67dffb640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa66c06b520 0x7fa66c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.427+0000 7fa68d368640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa66c06b520 0x7fa66c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:25.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.427+0000 7fa68d368640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa66c06b520 0x7fa66c06d9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa680009aa0 tx=0x7fa680009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:25.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.429+0000 7fa67dffb640 1 -- 172.21.15.133:0/4171300192 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa6840057a0 con 0x7fa688150d30 2024-02-21T23:25:25.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.666+0000 7fa67dffb640 1 -- 172.21.15.133:0/4171300192 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fa6840591f0 con 0x7fa688150d30 2024-02-21T23:25:25.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.838+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6880c9750 con 0x7fa688150d30 2024-02-21T23:25:25.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.876+0000 7fa67dffb640 1 -- 172.21.15.133:0/4171300192 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fa68405f070 con 0x7fa688150d30 2024-02-21T23:25:25.864 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:25.864 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa66c06b520 msgr2=0x7fa66c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa66c06b520 0x7fa66c06d9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa680009aa0 tx=0x7fa680009190 comp rx=0 tx=0).stop 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688150d30 msgr2=0x7fa6881f1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688150d30 0x7fa6881f1460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c690 tx=0x7fa68400cb50 comp rx=0 tx=0).stop 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 shutdown_connections 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa66c06b520 0x7fa66c06d9e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688150d30 0x7fa6881f1460 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6880c41a0 0x7fa68814ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 --2- 172.21.15.133:0/4171300192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6880c3dc0 0x7fa6881f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 >> 172.21.15.133:0/4171300192 conn(0x7fa688061b50 msgr2=0x7fa68806bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 shutdown_connections 2024-02-21T23:25:25.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:25.879+0000 7fa68f5cc640 1 -- 172.21.15.133:0/4171300192 wait complete. 2024-02-21T23:25:26.002 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:25:26.005 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:25:26.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-21T23:25:16.775942+0000', 'modified': '2024-02-21T23:25:22.297333+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4269}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4269': {'gid': 4269, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.142:6835/1527124199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 1527124199}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 1527124199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-02-21T23:25:26.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-21T23:25:26.481 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-02-21T23:25:26.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.598+0000 7ff54c55d640 1 Processor -- start 2024-02-21T23:25:26.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.598+0000 7ff54c55d640 1 -- start start 2024-02-21T23:25:26.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.599+0000 7ff54c55d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff544150a30 0x7ff5440c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.599+0000 7ff54c55d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff544071f60 0x7ff5440c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.599+0000 7ff54c55d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5440c53e0 con 0x7ff544071150 2024-02-21T23:25:26.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.599+0000 7ff54c55d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff544072540 con 0x7ff544071f60 2024-02-21T23:25:26.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.599+0000 7ff54c55d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5440c5680 con 0x7ff544150a30 2024-02-21T23:25:26.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.599+0000 7ff549af8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff544071f60 0x7ff5440c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.599+0000 7ff54a2f9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff544150a30 0x7ff5440c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.600+0000 7ff54aafa640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7ff544071150 0x7ff54406e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:40338/0 (socket says 172.21.15.133:40338) 2024-02-21T23:25:26.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.600+0000 7ff54aafa640 1 -- 172.21.15.133:0/2701123411 learned_addr learned my addr 172.21.15.133:0/2701123411 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:26.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.600+0000 7ff549af8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff544071f60 0x7ff5440c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48962/0 (socket says 172.21.15.133:48962) 2024-02-21T23:25:26.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.602+0000 7ff549af8640 1 -- 172.21.15.133:0/2701123411 >> v2:172.21.15.142:3300/0 conn(0x7ff544150a30 msgr2=0x7ff5440c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:26.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.602+0000 7ff549af8640 1 --2- 172.21.15.133:0/2701123411 >> v2:172.21.15.142:3300/0 conn(0x7ff544150a30 0x7ff5440c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.602+0000 7ff549af8640 1 -- 172.21.15.133:0/2701123411 >> v1:172.21.15.142:6789/0 conn(0x7ff544071150 legacy=0x7ff54406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.602+0000 7ff549af8640 1 -- 172.21.15.133:0/2701123411 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5440c5960 con 0x7ff544071f60 2024-02-21T23:25:26.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.604+0000 7ff549af8640 1 --2- 172.21.15.133:0/2701123411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff544071f60 0x7ff5440c7ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff54406be20 tx=0x7ff53402df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73b649c5baafa79 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.607+0000 7ff5492f7640 1 -- 172.21.15.133:0/2701123411 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53403b070 con 0x7ff544071f60 2024-02-21T23:25:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.607+0000 7ff5492f7640 1 -- 172.21.15.133:0/2701123411 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53402ece0 con 0x7ff544071f60 2024-02-21T23:25:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.607+0000 7ff5492f7640 1 -- 172.21.15.133:0/2701123411 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53400b040 con 0x7ff544071f60 2024-02-21T23:25:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.608+0000 7ff54c55d640 1 -- 172.21.15.133:0/2701123411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff544071f60 msgr2=0x7ff5440c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.608+0000 7ff54c55d640 1 --2- 172.21.15.133:0/2701123411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff544071f60 0x7ff5440c7ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff54406be20 tx=0x7ff53402df30 comp rx=0 tx=0).stop 2024-02-21T23:25:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.608+0000 7ff54c55d640 1 -- 172.21.15.133:0/2701123411 shutdown_connections 2024-02-21T23:25:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.608+0000 7ff54c55d640 1 --2- 172.21.15.133:0/2701123411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff544071f60 0x7ff5440c7ac0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.608+0000 7ff54c55d640 1 --2- 172.21.15.133:0/2701123411 >> v2:172.21.15.142:3300/0 conn(0x7ff544150a30 0x7ff5440c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.608+0000 7ff54c55d640 1 -- 172.21.15.133:0/2701123411 >> 172.21.15.133:0/2701123411 conn(0x7ff54406aec0 msgr2=0x7ff54406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.608+0000 7ff54c55d640 1 -- 172.21.15.133:0/2701123411 shutdown_connections 2024-02-21T23:25:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.608+0000 7ff54c55d640 1 -- 172.21.15.133:0/2701123411 wait complete. 2024-02-21T23:25:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.609+0000 7ff54c55d640 1 Processor -- start 2024-02-21T23:25:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.609+0000 7ff54c55d640 1 -- start start 2024-02-21T23:25:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.610+0000 7ff54c55d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff544071150 0x7ff544143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.610+0000 7ff54c55d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff544071f60 0x7ff54414a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.610+0000 7ff54c55d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff544150a30 0x7ff54414f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.610+0000 7ff54c55d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff544150e10 con 0x7ff544071f60 2024-02-21T23:25:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.610+0000 7ff54c55d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff544072a70 con 0x7ff544150a30 2024-02-21T23:25:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.610+0000 7ff54c55d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff544150f90 con 0x7ff544071150 2024-02-21T23:25:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.611+0000 7ff54aafa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff544150a30 0x7ff54414f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.611+0000 7ff54a2f9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff544071150 0x7ff544143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.611+0000 7ff54aafa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff544150a30 0x7ff54414f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56758/0 (socket says 172.21.15.133:56758) 2024-02-21T23:25:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.611+0000 7ff54aafa640 1 -- 172.21.15.133:0/2507223677 learned_addr learned my addr 172.21.15.133:0/2507223677 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:26.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.614+0000 7ff54aafa640 1 -- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff544071150 msgr2=0x7ff544143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.614+0000 7ff54aafa640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff544071150 0x7ff544143880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.614+0000 7ff54aafa640 1 -- 172.21.15.133:0/2507223677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff544071f60 msgr2=0x7ff54414a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.614+0000 7ff54aafa640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff544071f60 0x7ff54414a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.614+0000 7ff54aafa640 1 -- 172.21.15.133:0/2507223677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5441ebb50 con 0x7ff544150a30 2024-02-21T23:25:26.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.617+0000 7ff54aafa640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff544150a30 0x7ff54414f640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00cb00 tx=0x7ff53c00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:26.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.620+0000 7ff53affd640 1 -- 172.21.15.133:0/2507223677 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c017070 con 0x7ff544150a30 2024-02-21T23:25:26.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.620+0000 7ff53affd640 1 -- 172.21.15.133:0/2507223677 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c00aca0 con 0x7ff544150a30 2024-02-21T23:25:26.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.620+0000 7ff54c55d640 1 -- 172.21.15.133:0/2507223677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff544145440 con 0x7ff544150a30 2024-02-21T23:25:26.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.620+0000 7ff53affd640 1 -- 172.21.15.133:0/2507223677 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c012620 con 0x7ff544150a30 2024-02-21T23:25:26.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.620+0000 7ff54c55d640 1 -- 172.21.15.133:0/2507223677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff544145a00 con 0x7ff544150a30 2024-02-21T23:25:26.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.624+0000 7ff54f418740 1 Processor -- start 2024-02-21T23:25:26.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.625+0000 7ff54f418740 1 -- start start 2024-02-21T23:25:26.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.625+0000 7ff54f418740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.625+0000 7ff54f418740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0c7f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.625+0000 7ff54f418740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7d550 0x5594a0cbe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.625+0000 7ff54f418740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a0be2860 con 0x5594a0c7eaf0 2024-02-21T23:25:26.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.625+0000 7ff54f418740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a0b8b980 con 0x5594a0cc8970 2024-02-21T23:25:26.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.625+0000 7ff54f418740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a0bdeaa0 con 0x5594a0c7d550 2024-02-21T23:25:26.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.626+0000 7ff549af8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7d550 0x5594a0cbe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.626+0000 7ff549af8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7d550 0x5594a0cbe470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:56042/0 (socket says 172.21.15.133:56042) 2024-02-21T23:25:26.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.626+0000 7ff549af8640 1 -- 172.21.15.133:0/2796609980 learned_addr learned my addr 172.21.15.133:0/2796609980 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:26.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.628+0000 7ff549af8640 1 --2- 172.21.15.133:0/2796609980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c7eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.628+0000 7ff54a2f9640 1 --2- 172.21.15.133:0/2796609980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0c7f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.629+0000 7ff549af8640 1 -- 172.21.15.133:0/2796609980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 msgr2=0x5594a0c7f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:26.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.629+0000 7ff549af8640 1 --2- 172.21.15.133:0/2796609980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0c7f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.629+0000 7ff549af8640 1 -- 172.21.15.133:0/2796609980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 msgr2=0x5594a0c7eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:26.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.629+0000 7ff549af8640 1 --2- 172.21.15.133:0/2796609980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c7eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.629+0000 7ff549af8640 1 -- 172.21.15.133:0/2796609980 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a0ba0c60 con 0x5594a0c7d550 2024-02-21T23:25:26.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.631+0000 7ff549af8640 1 --2- 172.21.15.133:0/2796609980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7d550 0x5594a0cbe470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff534000c00 tx=0x7ff534030070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4406cdeda5595b87 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:26.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.635+0000 7ff527fff640 1 -- 172.21.15.133:0/2796609980 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534048070 con 0x5594a0c7d550 2024-02-21T23:25:26.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.635+0000 7ff527fff640 1 -- 172.21.15.133:0/2796609980 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53402ed00 con 0x5594a0c7d550 2024-02-21T23:25:26.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.635+0000 7ff527fff640 1 -- 172.21.15.133:0/2796609980 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53403b040 con 0x5594a0c7d550 2024-02-21T23:25:26.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.636+0000 7ff54f418740 1 -- 172.21.15.133:0/2796609980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7d550 msgr2=0x5594a0cbe470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.636+0000 7ff54f418740 1 --2- 172.21.15.133:0/2796609980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7d550 0x5594a0cbe470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff534000c00 tx=0x7ff534030070 comp rx=0 tx=0).stop 2024-02-21T23:25:26.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.636+0000 7ff54f418740 1 -- 172.21.15.133:0/2796609980 shutdown_connections 2024-02-21T23:25:26.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.636+0000 7ff54f418740 1 --2- 172.21.15.133:0/2796609980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7d550 0x5594a0cbe470 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.636+0000 7ff54f418740 1 --2- 172.21.15.133:0/2796609980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0c7f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.636+0000 7ff54f418740 1 --2- 172.21.15.133:0/2796609980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c7eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.636+0000 7ff54f418740 1 -- 172.21.15.133:0/2796609980 >> 172.21.15.133:0/2796609980 conn(0x5594a0c7b260 msgr2=0x5594a0c7dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:26.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.636+0000 7ff54f418740 1 -- 172.21.15.133:0/2796609980 shutdown_connections 2024-02-21T23:25:26.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.636+0000 7ff54f418740 1 -- 172.21.15.133:0/2796609980 wait complete. 2024-02-21T23:25:26.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.637+0000 7ff54f418740 1 -- start start 2024-02-21T23:25:26.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.637+0000 7ff54f418740 1 Processor -- start 2024-02-21T23:25:26.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff54f418740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0cd2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff54f418740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0cc2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff54f418740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 0x5594a0ca1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff54f418740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a0b9ef10 con 0x5594a0c7eaf0 2024-02-21T23:25:26.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff54f418740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a0cc8eb0 con 0x5594a0cc8970 2024-02-21T23:25:26.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff54f418740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a0cd33e0 con 0x5594a0cbb6e0 2024-02-21T23:25:26.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff54f418740 10 client.0 _ll_register_callbacks cb 0x5594a0b92d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-21T23:25:26.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff549af8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0cd2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff549af8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 0x5594a0ca1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff549af8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0cd2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48980/0 (socket says 172.21.15.133:48980) 2024-02-21T23:25:26.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.638+0000 7ff549af8640 1 -- 172.21.15.133:0/3656108466 learned_addr learned my addr 172.21.15.133:0/3656108466 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:26.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.641+0000 7ff549af8640 1 -- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 msgr2=0x5594a0ca1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:26.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.641+0000 7ff549af8640 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 0x5594a0ca1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.641+0000 7ff549af8640 1 -- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 msgr2=0x5594a0cc2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:26.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.641+0000 7ff549af8640 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0cc2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.641+0000 7ff549af8640 1 -- 172.21.15.133:0/3656108466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594a08f72d0 con 0x5594a0c7eaf0 2024-02-21T23:25:26.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.643+0000 7ff53affd640 1 -- 172.21.15.133:0/2507223677 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff53c01a070 con 0x7ff544150a30 2024-02-21T23:25:26.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.643+0000 7ff53affd640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff52806b490 0x7ff52806d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.643+0000 7ff54a2f9640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff52806b490 0x7ff52806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.644+0000 7ff549af8640 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0cd2a70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff53400aa30 tx=0x7ff534030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bc0f4e7c3344b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:26.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.644+0000 7ff5257fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-02-21T23:25:26.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.644+0000 7ff54a2f9640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff52806b490 0x7ff52806d950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff540009a40 tx=0x7ff540008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:26.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.646+0000 7ff53affd640 1 -- 172.21.15.133:0/2507223677 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff53c056780 con 0x7ff544150a30 2024-02-21T23:25:26.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.647+0000 7ff5257fa640 1 -- 172.21.15.133:0/3656108466 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53403b070 con 0x5594a0c7eaf0 2024-02-21T23:25:26.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.647+0000 7ff5257fa640 1 -- 172.21.15.133:0/3656108466 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff534054e00 con 0x5594a0c7eaf0 2024-02-21T23:25:26.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.647+0000 7ff54f418740 1 -- 172.21.15.133:0/3656108466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff53402ed00 con 0x5594a0c7eaf0 2024-02-21T23:25:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.649+0000 7ff5257fa640 1 -- 172.21.15.133:0/3656108466 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff534048030 con 0x5594a0c7eaf0 2024-02-21T23:25:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.649+0000 7ff54f418740 10 client.4356 fetch_fsmap learned FSMap version 7 2024-02-21T23:25:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.649+0000 7ff54f418740 1 -- 172.21.15.133:0/3656108466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5594a09c16b0 con 0x5594a0c7eaf0 2024-02-21T23:25:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.653+0000 7ff5257fa640 1 -- 172.21.15.133:0/3656108466 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53402ed00 con 0x5594a0c7eaf0 2024-02-21T23:25:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.653+0000 7ff5257fa640 1 -- 172.21.15.133:0/3656108466 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 7) v1 ==== 4452+0+0 (secure 0 0 0) 0x7ff534041040 con 0x5594a0c7eaf0 2024-02-21T23:25:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.653+0000 7ff54f418740 10 client.4356 fetch_fsmap finished waiting for FSMap version 7 2024-02-21T23:25:26.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.653+0000 7ff54f418740 10 client.4356 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-21T23:25:26.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.653+0000 7ff54f418740 20 client.4356 populate_metadata read hostname 'smithi133' 2024-02-21T23:25:26.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.653+0000 7ff54f418740 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x5594a0cb9990 0x5594a0cde110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.653+0000 7ff54f418740 4 client.4356 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-21T23:25:26.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.653+0000 7ff54f418740 1 -- 172.21.15.133:0/3656108466 --> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594a0c40480 con 0x5594a0cb9990 2024-02-21T23:25:26.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.662+0000 7ff549af8640 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x5594a0cb9990 0x5594a0cde110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.755+0000 7ff549af8640 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x5594a0cb9990 0x5594a0cde110 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73b1fe2b86beec server_cookie=2b86a5ff7bde6a4d in_seq=0 out_seq=0 2024-02-21T23:25:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.755+0000 7ff5257fa640 10 client.4356 ms_handle_connect on v2:172.21.15.142:6834/1527124199 2024-02-21T23:25:26.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff5257fa640 1 -- 172.21.15.133:0/3656108466 <== mds.0 v2:172.21.15.142:6834/1527124199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5594a0c40480 con 0x5594a0cb9990 2024-02-21T23:25:26.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff5257fa640 10 client.4356 handle_command_reply: tid=0 2024-02-21T23:25:26.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 client.4356 shutdown 2024-02-21T23:25:26.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 -- 172.21.15.133:0/3656108466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 msgr2=0x5594a0cd2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0cd2a70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff53400aa30 tx=0x7ff534030000 comp rx=0 tx=0).stop 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 -- 172.21.15.133:0/3656108466 shutdown_connections 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 0x5594a0ca1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0cc2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x5594a0cb9990 0x5594a0cde110 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 --2- 172.21.15.133:0/3656108466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0cd2a70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff5257fa640 0 client.4356 ms_handle_reset on v2:172.21.15.142:6834/1527124199 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 -- 172.21.15.133:0/3656108466 >> 172.21.15.133:0/3656108466 conn(0x5594a0c7d550 msgr2=0x5594a0c78000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 -- 172.21.15.133:0/3656108466 shutdown_connections 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 1 -- 172.21.15.133:0/3656108466 wait complete. 2024-02-21T23:25:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.876+0000 7ff54f418740 20 client.4356 trim_cache size 0 max 16384 2024-02-21T23:25:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.894+0000 7ff54f418740 1 Processor -- start 2024-02-21T23:25:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.894+0000 7ff54f418740 1 -- start start 2024-02-21T23:25:26.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.894+0000 7ff54f418740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c79cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.894+0000 7ff54f418740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 0x5594a0c7bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.894+0000 7ff54f418740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0c7c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.894+0000 7ff54f418740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5441f8520 con 0x5594a0c7eaf0 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.894+0000 7ff54f418740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a0b9ef10 con 0x5594a0cc8970 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.894+0000 7ff54f418740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a0cbaa40 con 0x5594a0cbb6e0 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.895+0000 7ff54a2f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c79cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.895+0000 7ff54aafa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0c7c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.895+0000 7ff549af8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 0x5594a0c7bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.895+0000 7ff54a2f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c79cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48984/0 (socket says 172.21.15.133:48984) 2024-02-21T23:25:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.895+0000 7ff54a2f9640 1 -- 172.21.15.133:0/2502039045 learned_addr learned my addr 172.21.15.133:0/2502039045 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.897+0000 7ff54a2f9640 1 -- 172.21.15.133:0/2502039045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 msgr2=0x5594a0c7bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.897+0000 7ff54a2f9640 1 --2- 172.21.15.133:0/2502039045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 0x5594a0c7bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.897+0000 7ff54a2f9640 1 -- 172.21.15.133:0/2502039045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 msgr2=0x5594a0c7c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.897+0000 7ff54a2f9640 1 --2- 172.21.15.133:0/2502039045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0c7c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.897+0000 7ff54a2f9640 1 -- 172.21.15.133:0/2502039045 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a0cbd220 con 0x5594a0c7eaf0 2024-02-21T23:25:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.900+0000 7ff54a2f9640 1 --2- 172.21.15.133:0/2502039045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c79cb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff540009330 tx=0x7ff540016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6344afec57e24c49 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:26.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.903+0000 7ff5277fe640 1 -- 172.21.15.133:0/2502039045 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54001f050 con 0x5594a0c7eaf0 2024-02-21T23:25:26.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.903+0000 7ff5277fe640 1 -- 172.21.15.133:0/2502039045 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff540029070 con 0x5594a0c7eaf0 2024-02-21T23:25:26.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.903+0000 7ff5277fe640 1 -- 172.21.15.133:0/2502039045 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540024680 con 0x5594a0c7eaf0 2024-02-21T23:25:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 -- 172.21.15.133:0/2502039045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 msgr2=0x5594a0c79cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 --2- 172.21.15.133:0/2502039045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c79cb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff540009330 tx=0x7ff540016000 comp rx=0 tx=0).stop 2024-02-21T23:25:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 -- 172.21.15.133:0/2502039045 shutdown_connections 2024-02-21T23:25:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 --2- 172.21.15.133:0/2502039045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cc8970 0x5594a0c7c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 --2- 172.21.15.133:0/2502039045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0cbb6e0 0x5594a0c7bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 --2- 172.21.15.133:0/2502039045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0c7eaf0 0x5594a0c79cb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 -- 172.21.15.133:0/2502039045 >> 172.21.15.133:0/2502039045 conn(0x5594a0c7d550 msgr2=0x5594a0c78000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 -- 172.21.15.133:0/2502039045 shutdown_connections 2024-02-21T23:25:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 -- 172.21.15.133:0/2502039045 wait complete. 2024-02-21T23:25:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.904+0000 7ff54f418740 1 -- start start 2024-02-21T23:25:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.905+0000 7ff54f418740 1 Processor -- start 2024-02-21T23:25:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54f418740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7eaf0 0x5594a0c9b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54f418740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cbb6e0 0x5594a0cc17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54f418740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0cc8970 0x5594a0cc1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54f418740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a0ca6db0 con 0x5594a0cc8970 2024-02-21T23:25:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54f418740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a0ca2910 con 0x5594a0cbb6e0 2024-02-21T23:25:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54f418740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a0cb9e10 con 0x5594a0c7eaf0 2024-02-21T23:25:26.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54f418740 10 client.0 _ll_register_callbacks cb 0x7ff534046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-21T23:25:26.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54aafa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0cc8970 0x5594a0cc1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff549af8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cbb6e0 0x5594a0cc17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54a2f9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7eaf0 0x5594a0c9b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54aafa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0cc8970 0x5594a0cc1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48996/0 (socket says 172.21.15.133:48996) 2024-02-21T23:25:26.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.906+0000 7ff54aafa640 1 -- 172.21.15.133:0/398841445 learned_addr learned my addr 172.21.15.133:0/398841445 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.909+0000 7ff54aafa640 1 -- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7eaf0 msgr2=0x5594a0c9b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.909+0000 7ff54aafa640 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7eaf0 0x5594a0c9b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.909+0000 7ff54aafa640 1 -- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cbb6e0 msgr2=0x5594a0cc17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.909+0000 7ff54aafa640 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cbb6e0 0x5594a0cc17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.909+0000 7ff54aafa640 1 -- 172.21.15.133:0/398841445 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594a0ccd220 con 0x5594a0cc8970 2024-02-21T23:25:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.909+0000 7ff549af8640 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cbb6e0 0x5594a0cc17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:25:26.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.912+0000 7ff54aafa640 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0cc8970 0x5594a0cc1ce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff53c058670 tx=0x7ff53c058c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15ad5ec1fbc4ceb server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:26.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.912+0000 7ff5257fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-02-21T23:25:26.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.915+0000 7ff5257fa640 1 -- 172.21.15.133:0/398841445 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c057e00 con 0x5594a0cc8970 2024-02-21T23:25:26.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.915+0000 7ff5257fa640 1 -- 172.21.15.133:0/398841445 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff53c054430 con 0x5594a0cc8970 2024-02-21T23:25:26.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.915+0000 7ff54f418740 1 -- 172.21.15.133:0/398841445 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff540029070 con 0x5594a0cc8970 2024-02-21T23:25:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.917+0000 7ff5257fa640 1 -- 172.21.15.133:0/398841445 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53c075050 con 0x5594a0cc8970 2024-02-21T23:25:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.917+0000 7ff54f418740 10 client.4362 fetch_fsmap learned FSMap version 7 2024-02-21T23:25:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.917+0000 7ff54f418740 1 -- 172.21.15.133:0/398841445 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5594a0cbc380 con 0x5594a0cc8970 2024-02-21T23:25:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.921+0000 7ff5257fa640 1 -- 172.21.15.133:0/398841445 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff540029070 con 0x5594a0cc8970 2024-02-21T23:25:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.921+0000 7ff5257fa640 1 -- 172.21.15.133:0/398841445 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 7) v1 ==== 4452+0+0 (secure 0 0 0) 0x7ff53c0575a0 con 0x5594a0cc8970 2024-02-21T23:25:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.921+0000 7ff54f418740 10 client.4362 fetch_fsmap finished waiting for FSMap version 7 2024-02-21T23:25:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.921+0000 7ff54f418740 10 client.4362 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-21T23:25:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.921+0000 7ff54f418740 20 client.4362 populate_metadata read hostname 'smithi133' 2024-02-21T23:25:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.921+0000 7ff54f418740 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x5594a0d2e720 0x5594a0d2eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.921+0000 7ff54f418740 4 client.4362 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-21T23:25:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.921+0000 7ff54f418740 1 -- 172.21.15.133:0/398841445 --> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5594a0befda0 con 0x5594a0d2e720 2024-02-21T23:25:26.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.922+0000 7ff54a2f9640 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x5594a0d2e720 0x5594a0d2eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:26.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.925+0000 7ff54a2f9640 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x5594a0d2e720 0x5594a0d2eb00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1237f345e94880 server_cookie=57e9fe3226b8b2e0 in_seq=0 out_seq=0 2024-02-21T23:25:26.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.925+0000 7ff5257fa640 10 client.4362 ms_handle_connect on v2:172.21.15.142:6834/1527124199 2024-02-21T23:25:26.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff5257fa640 1 -- 172.21.15.133:0/398841445 <== mds.0 v2:172.21.15.142:6834/1527124199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+316 (crc 0 0 0) 0x5594a0befda0 con 0x5594a0d2e720 2024-02-21T23:25:26.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff5257fa640 10 client.4362 handle_command_reply: tid=0 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 client.4362 shutdown 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 -- 172.21.15.133:0/398841445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0cc8970 msgr2=0x5594a0cc1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0cc8970 0x5594a0cc1ce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff53c058670 tx=0x7ff53c058c30 comp rx=0 tx=0).stop 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 -- 172.21.15.133:0/398841445 shutdown_connections 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x5594a0d2e720 0x5594a0d2eb00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594a0cc8970 0x5594a0cc1ce0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594a0cbb6e0 0x5594a0cc17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff5257fa640 0 client.4362 ms_handle_reset on v2:172.21.15.142:6834/1527124199 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 --2- 172.21.15.133:0/398841445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5594a0c7eaf0 0x5594a0c9b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 -- 172.21.15.133:0/398841445 >> 172.21.15.133:0/398841445 conn(0x5594a0c7d550 msgr2=0x5594a0cbbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 -- 172.21.15.133:0/398841445 shutdown_connections 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 1 -- 172.21.15.133:0/398841445 wait complete. 2024-02-21T23:25:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.944+0000 7ff54f418740 20 client.4362 trim_cache size 0 max 16384 2024-02-21T23:25:26.933 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:25:26.933 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "1f1c3b6d-6263-48c3-85b7-ccc808770c4d", 2024-02-21T23:25:26.933 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-02-21T23:25:26.933 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4269, 2024-02-21T23:25:26.933 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-02-21T23:25:26.933 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-02-21T23:25:26.933 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-02-21T23:25:26.934 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 9.565003935, 2024-02-21T23:25:26.934 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 7, 2024-02-21T23:25:26.934 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 24, 2024-02-21T23:25:26.934 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 23, 2024-02-21T23:25:26.934 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 16.92281015 2024-02-21T23:25:26.934 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:25:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 -- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff52806b490 msgr2=0x7ff52806d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff52806b490 0x7ff52806d950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff540009a40 tx=0x7ff540008040 comp rx=0 tx=0).stop 2024-02-21T23:25:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 -- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff544150a30 msgr2=0x7ff54414f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff544150a30 0x7ff54414f640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00cb00 tx=0x7ff53c00a060 comp rx=0 tx=0).stop 2024-02-21T23:25:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 -- 172.21.15.133:0/2507223677 shutdown_connections 2024-02-21T23:25:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff52806b490 0x7ff52806d950 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff544150a30 0x7ff54414f640 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff544071f60 0x7ff54414a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 --2- 172.21.15.133:0/2507223677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff544071150 0x7ff544143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 -- 172.21.15.133:0/2507223677 >> 172.21.15.133:0/2507223677 conn(0x7ff54406aec0 msgr2=0x7ff54406ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 -- 172.21.15.133:0/2507223677 shutdown_connections 2024-02-21T23:25:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:26.945+0000 7ff54c55d640 1 -- 172.21.15.133:0/2507223677 wait complete. 2024-02-21T23:25:26.942 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-21T23:25:26.942 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-21T23:25:26.943 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-21T23:25:27.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.055+0000 7fe793bfa640 1 Processor -- start 2024-02-21T23:25:27.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.055+0000 7fe793bfa640 1 -- start start 2024-02-21T23:25:27.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.056+0000 7fe793bfa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe78c0cd900 0x7fe78c0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:27.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.056+0000 7fe793bfa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c1588f0 0x7fe78c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:27.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.056+0000 7fe793bfa640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78c15bc10 con 0x7fe78c0c9180 2024-02-21T23:25:27.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.056+0000 7fe793bfa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c158460 con 0x7fe78c0cd900 2024-02-21T23:25:27.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.056+0000 7fe793bfa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c15bef0 con 0x7fe78c1588f0 2024-02-21T23:25:27.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.057+0000 7fe791195640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fe78c0c9180 0x7fe78c15b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:40344/0 (socket says 172.21.15.133:40344) 2024-02-21T23:25:27.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.057+0000 7fe791195640 1 -- 172.21.15.133:0/1306768598 learned_addr learned my addr 172.21.15.133:0/1306768598 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:27.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.057+0000 7fe791996640 1 --2- 172.21.15.133:0/1306768598 >> v2:172.21.15.133:3300/0 conn(0x7fe78c0cd900 0x7fe78c0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:27.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.057+0000 7fe792197640 1 --2- 172.21.15.133:0/1306768598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c1588f0 0x7fe78c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:27.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.059+0000 7fe791996640 1 -- 172.21.15.133:0/1306768598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c1588f0 msgr2=0x7fe78c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:27.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.059+0000 7fe791996640 1 --2- 172.21.15.133:0/1306768598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c1588f0 0x7fe78c0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.059+0000 7fe791996640 1 -- 172.21.15.133:0/1306768598 >> v1:172.21.15.142:6789/0 conn(0x7fe78c0c9180 legacy=0x7fe78c15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:27.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.059+0000 7fe791996640 1 -- 172.21.15.133:0/1306768598 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c15c1d0 con 0x7fe78c0cd900 2024-02-21T23:25:27.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.061+0000 7fe791996640 1 --2- 172.21.15.133:0/1306768598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 0x7fe78c0c9cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe788009d90 tx=0x7fe78802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c6dfb803fb3099 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:27.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.065+0000 7fe790994640 1 -- 172.21.15.133:0/1306768598 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788039070 con 0x7fe78c0cd900 2024-02-21T23:25:27.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.065+0000 7fe790994640 1 -- 172.21.15.133:0/1306768598 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78802cc50 con 0x7fe78c0cd900 2024-02-21T23:25:27.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.065+0000 7fe790994640 1 -- 172.21.15.133:0/1306768598 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788034440 con 0x7fe78c0cd900 2024-02-21T23:25:27.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.065+0000 7fe793bfa640 1 -- 172.21.15.133:0/1306768598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 msgr2=0x7fe78c0c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:27.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.065+0000 7fe793bfa640 1 --2- 172.21.15.133:0/1306768598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 0x7fe78c0c9cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe788009d90 tx=0x7fe78802bea0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.066+0000 7fe793bfa640 1 -- 172.21.15.133:0/1306768598 shutdown_connections 2024-02-21T23:25:27.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.066+0000 7fe793bfa640 1 --2- 172.21.15.133:0/1306768598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c1588f0 0x7fe78c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.066+0000 7fe793bfa640 1 --2- 172.21.15.133:0/1306768598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 0x7fe78c0c9cc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.066+0000 7fe793bfa640 1 -- 172.21.15.133:0/1306768598 >> 172.21.15.133:0/1306768598 conn(0x7fe78c06f7f0 msgr2=0x7fe78c061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:27.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.066+0000 7fe793bfa640 1 -- 172.21.15.133:0/1306768598 shutdown_connections 2024-02-21T23:25:27.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.066+0000 7fe793bfa640 1 -- 172.21.15.133:0/1306768598 wait complete. 2024-02-21T23:25:27.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.067+0000 7fe793bfa640 1 Processor -- start 2024-02-21T23:25:27.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.067+0000 7fe793bfa640 1 -- start start 2024-02-21T23:25:27.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe793bfa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c0c9180 0x7fe78c14c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:27.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe793bfa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 0x7fe78c1f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:27.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe793bfa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c1588f0 0x7fe78c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:27.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe793bfa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c0b3a10 con 0x7fe78c0cd900 2024-02-21T23:25:27.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe793bfa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c1580f0 con 0x7fe78c1588f0 2024-02-21T23:25:27.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe793bfa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c0b3b90 con 0x7fe78c0c9180 2024-02-21T23:25:27.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe791195640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 0x7fe78c1f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:27.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe792197640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c1588f0 0x7fe78c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:27.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe791996640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c0c9180 0x7fe78c14c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:27.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe792197640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c1588f0 0x7fe78c156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56808/0 (socket says 172.21.15.133:56808) 2024-02-21T23:25:27.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.068+0000 7fe792197640 1 -- 172.21.15.133:0/1936869111 learned_addr learned my addr 172.21.15.133:0/1936869111 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:27.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.072+0000 7fe792197640 1 -- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c0c9180 msgr2=0x7fe78c14c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:27.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.072+0000 7fe792197640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c0c9180 0x7fe78c14c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.072+0000 7fe792197640 1 -- 172.21.15.133:0/1936869111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 msgr2=0x7fe78c1f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:27.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.072+0000 7fe792197640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 0x7fe78c1f1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.072+0000 7fe792197640 1 -- 172.21.15.133:0/1936869111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c14d0c0 con 0x7fe78c1588f0 2024-02-21T23:25:27.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.072+0000 7fe791195640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 0x7fe78c1f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:25:27.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.074+0000 7fe792197640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c1588f0 0x7fe78c156d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe78400e850 tx=0x7fe78400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:27.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.079+0000 7fe7827fc640 1 -- 172.21.15.133:0/1936869111 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784017070 con 0x7fe78c1588f0 2024-02-21T23:25:27.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.079+0000 7fe7827fc640 1 -- 172.21.15.133:0/1936869111 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78400c940 con 0x7fe78c1588f0 2024-02-21T23:25:27.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.079+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c14d3a0 con 0x7fe78c1588f0 2024-02-21T23:25:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.080+0000 7fe7827fc640 1 -- 172.21.15.133:0/1936869111 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78400eea0 con 0x7fe78c1588f0 2024-02-21T23:25:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.080+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c1ee420 con 0x7fe78c1588f0 2024-02-21T23:25:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.081+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78c1edf30 con 0x7fe78c1588f0 2024-02-21T23:25:27.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.112+0000 7fe7827fc640 1 -- 172.21.15.133:0/1936869111 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe78401a070 con 0x7fe78c1588f0 2024-02-21T23:25:27.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.112+0000 7fe7827fc640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe77006b4d0 0x7fe77006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:27.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.113+0000 7fe791996640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe77006b4d0 0x7fe77006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.113+0000 7fe791996640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe77006b4d0 0x7fe77006d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe788036000 tx=0x7fe788030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.117+0000 7fe7827fc640 1 -- 172.21.15.133:0/1936869111 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe7840570b0 con 0x7fe78c1588f0 2024-02-21T23:25:27.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.370+0000 7fe7827fc640 1 -- 172.21.15.133:0/1936869111 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fe784059010 con 0x7fe78c1588f0 2024-02-21T23:25:27.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.584+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe78c06a130 con 0x7fe78c1588f0 2024-02-21T23:25:27.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.603+0000 7fe7827fc640 1 -- 172.21.15.133:0/1936869111 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe78405d0b0 con 0x7fe78c1588f0 2024-02-21T23:25:27.590 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.605+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe77006b4d0 msgr2=0x7fe77006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.605+0000 7fe793bfa640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe77006b4d0 0x7fe77006d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe788036000 tx=0x7fe788030000 comp rx=0 tx=0).stop 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.605+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c1588f0 msgr2=0x7fe78c156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.605+0000 7fe793bfa640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c1588f0 0x7fe78c156d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe78400e850 tx=0x7fe78400ec50 comp rx=0 tx=0).stop 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.605+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 shutdown_connections 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.605+0000 7fe793bfa640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe77006b4d0 0x7fe77006d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.606+0000 7fe793bfa640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe78c1588f0 0x7fe78c156d40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.606+0000 7fe793bfa640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c0cd900 0x7fe78c1f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.606+0000 7fe793bfa640 1 --2- 172.21.15.133:0/1936869111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe78c0c9180 0x7fe78c14c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.606+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 >> 172.21.15.133:0/1936869111 conn(0x7fe78c06f7f0 msgr2=0x7fe78c15a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.606+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 shutdown_connections 2024-02-21T23:25:27.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:27.606+0000 7fe793bfa640 1 -- 172.21.15.133:0/1936869111 wait complete. 2024-02-21T23:25:27.603 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-21T23:25:27.603 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-21T23:25:27.603 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-02-21T23:25:27.604 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-02-21T23:25:27.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-21T23:25:27.604 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-21T23:25:27.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-21T23:25:27.604 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-21T23:25:27.604 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-02-21T23:25:27.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:27.605 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-02-21T23:25:27.619 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-21T23:25:27.619 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-21T23:25:27.620 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-02-21T23:25:27.621 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-21T23:25:27.621 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41973sec preferred_lft 41973sec 2024-02-21T23:25:27.621 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-02-21T23:25:27.621 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-02-21T23:25:27.621 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-21T23:25:27.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:27.622 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:25:27.622 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:25:27.622 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2024-02-21T23:25:27.622 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2024-02-21T23:25:27.622 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2024-02-21T23:25:27.622 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-21T23:25:27.622 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:25:27.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:27.792 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-21T23:25:27.820 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-02-21T23:25:27.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:27.821 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-02-21T23:25:27.885 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-02-21T23:25:27.886 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-21T23:25:27.886 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-21T23:25:27.886 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-21T23:25:27.886 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-21T23:25:27.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:27.886 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:25:27.886 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:25:27.886 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-21T23:25:27.886 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-21T23:25:27.886 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-21T23:25:27.887 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:25:28.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:28.149 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-02-21T23:25:28.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:28.164 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-02-21T23:25:28.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:28.219 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:25:28.219 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:25:28.219 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-21T23:25:28.219 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-02-21T23:25:28.219 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:25:28.301 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-02-21T23:25:28.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:28.302 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:25:28.302 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:25:28.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-02-21T23:25:28.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-21T23:25:28.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-02-21T23:25:28.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-02-21T23:25:28.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-02-21T23:25:28.302 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:25:28.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:28.437 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:25:28.437 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:25:28.437 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-02-21T23:25:28.437 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2024-02-21T23:25:28.437 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:25:28.479 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-02-21T23:25:28.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-21T23:25:28.479 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-02-21T23:25:28.535 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-02-21T23:25:28.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-21T23:25:28.535 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-02-21T23:25:28.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:25:28.658 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:25:28.714 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:25:28.714 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:25:28.714 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:25:28.714 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:25:28.714 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:25:28.714 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:25:28.715 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-21T23:25:28.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:28.716 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:25:28.769 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-21T23:25:28.769 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-02-21T23:25:28.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:25:28.839 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:25:28.893 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:25:28.893 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:25:28.893 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:25:28.893 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:25:28.893 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:25:28.894 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-21T23:25:28.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:28.895 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:25:28.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:2024-02-21T23:25:28.959+0000 7f9909246480 -1 init, newargv = 0x55ea5084caf0 newargc=15 2024-02-21T23:25:28.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[42590]: starting ceph client 2024-02-21T23:25:29.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[42590]: starting fuse 2024-02-21T23:25:30.049 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:25:30.078 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:25:30.134 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:25:30.134 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:25:30.134 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:25:30.135 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-21T23:25:30.136 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-21T23:25:30.137 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.admin 2024-02-21T23:25:30.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:30.137 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:25:30.189 INFO:teuthology.orchestra.run.smithi133.stdout:41 2024-02-21T23:25:30.190 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-02-21T23:25:30.190 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.admin")) 2024-02-21T23:25:30.191 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-02-21T23:25:30.309 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.admin.42590.asok 2024-02-21T23:25:30.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-21T23:25:30.320 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42590.asok 2024-02-21T23:25:30.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:30.320 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42590.asok status 2024-02-21T23:25:30.457 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:25:30.457 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-02-21T23:25:30.457 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "3bfc837d7b45ce8e095d92fa9c342964cea5e95f", 2024-02-21T23:25:30.457 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1643-g3bfc837d (3bfc837d7b45ce8e095d92fa9c342964cea5e95f) squid (dev)", 2024-02-21T23:25:30.457 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "admin", 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "42590", 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4319, 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-02-21T23:25:30.458 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 4319 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1066198264 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1066198264 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.4319 172.21.15.133:0/1066198264", 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "172.21.15.133:0/1066198264", 2024-02-21T23:25:30.459 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-02-21T23:25:30.460 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 7, 2024-02-21T23:25:30.460 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 24, 2024-02-21T23:25:30.460 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-02-21T23:25:30.460 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-02-21T23:25:30.460 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-02-21T23:25:30.460 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:25:30.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:30.466 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-02-21T23:25:30.466 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-02-21T23:25:30.492 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-02-21T23:25:30.492 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-02-21T23:25:30.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:30.492 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-02-21T23:25:30.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:30.635 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-02-21T23:25:30.847 INFO:teuthology.orchestra.run.smithi133.stdout:# file: . 2024-02-21T23:25:30.847 INFO:teuthology.orchestra.run.smithi133.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-02-21T23:25:30.847 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:30.848 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:25:30.848 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-21T23:25:30.863 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:31.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.018+0000 7f91572ad640 1 Processor -- start 2024-02-21T23:25:31.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.018+0000 7f91572ad640 1 -- start start 2024-02-21T23:25:31.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.019+0000 7f91572ad640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f91500cdfe0 0x7f91500c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.019+0000 7f91572ad640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91500ce3c0 0x7f9150070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.019+0000 7f91572ad640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91500717e0 con 0x7f91500712b0 2024-02-21T23:25:31.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.019+0000 7f91572ad640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91500c84b0 con 0x7f91500ce3c0 2024-02-21T23:25:31.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.019+0000 7f91572ad640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9150158ca0 con 0x7f91500cdfe0 2024-02-21T23:25:31.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.020+0000 7f9154848640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91500ce3c0 0x7f9150070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.020+0000 7f9155049640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f91500cdfe0 0x7f91500c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.020+0000 7f915584a640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f91500712b0 0x7f915014e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:40354/0 (socket says 172.21.15.133:40354) 2024-02-21T23:25:31.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.020+0000 7f915584a640 1 -- 172.21.15.133:0/38111741 learned_addr learned my addr 172.21.15.133:0/38111741 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.022+0000 7f9154848640 1 -- 172.21.15.133:0/38111741 >> v2:172.21.15.142:3300/0 conn(0x7f91500cdfe0 msgr2=0x7f91500c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.022+0000 7f9154848640 1 --2- 172.21.15.133:0/38111741 >> v2:172.21.15.142:3300/0 conn(0x7f91500cdfe0 0x7f91500c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.022+0000 7f9154848640 1 -- 172.21.15.133:0/38111741 >> v1:172.21.15.142:6789/0 conn(0x7f91500712b0 legacy=0x7f915014e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.022+0000 7f9154848640 1 -- 172.21.15.133:0/38111741 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9150158090 con 0x7f91500ce3c0 2024-02-21T23:25:31.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.024+0000 7f9154848640 1 --2- 172.21.15.133:0/38111741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91500ce3c0 0x7f9150070d70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f915006b3b0 tx=0x7f914c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868fbe30f3e262bc server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f9147fff640 1 -- 172.21.15.133:0/38111741 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c035020 con 0x7f91500ce3c0 2024-02-21T23:25:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f9147fff640 1 -- 172.21.15.133:0/38111741 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c031830 con 0x7f91500ce3c0 2024-02-21T23:25:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f9147fff640 1 -- 172.21.15.133:0/38111741 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c031c00 con 0x7f91500ce3c0 2024-02-21T23:25:31.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f91572ad640 1 -- 172.21.15.133:0/38111741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91500ce3c0 msgr2=0x7f9150070d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f91572ad640 1 --2- 172.21.15.133:0/38111741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91500ce3c0 0x7f9150070d70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f915006b3b0 tx=0x7f914c02b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f91572ad640 1 -- 172.21.15.133:0/38111741 shutdown_connections 2024-02-21T23:25:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f91572ad640 1 --2- 172.21.15.133:0/38111741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91500ce3c0 0x7f9150070d70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f91572ad640 1 --2- 172.21.15.133:0/38111741 >> v2:172.21.15.142:3300/0 conn(0x7f91500cdfe0 0x7f91500c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f91572ad640 1 -- 172.21.15.133:0/38111741 >> 172.21.15.133:0/38111741 conn(0x7f915006aec0 msgr2=0x7f915006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f91572ad640 1 -- 172.21.15.133:0/38111741 shutdown_connections 2024-02-21T23:25:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.028+0000 7f91572ad640 1 -- 172.21.15.133:0/38111741 wait complete. 2024-02-21T23:25:31.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.030+0000 7f91572ad640 1 Processor -- start 2024-02-21T23:25:31.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.030+0000 7f91572ad640 1 -- start start 2024-02-21T23:25:31.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f91572ad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91500712b0 0x7f91501f0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f91572ad640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91500cdfe0 0x7f91501f1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f91572ad640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91500ce3c0 0x7f91501eee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f91572ad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9150158930 con 0x7f91500712b0 2024-02-21T23:25:31.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f91572ad640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9150158650 con 0x7f91500ce3c0 2024-02-21T23:25:31.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f91572ad640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915014ec20 con 0x7f91500cdfe0 2024-02-21T23:25:31.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f9155049640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91500712b0 0x7f91501f0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f915584a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91500ce3c0 0x7f91501eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f9154848640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91500cdfe0 0x7f91501f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f915584a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91500ce3c0 0x7f91501eee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56826/0 (socket says 172.21.15.133:56826) 2024-02-21T23:25:31.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.031+0000 7f915584a640 1 -- 172.21.15.133:0/1105630831 learned_addr learned my addr 172.21.15.133:0/1105630831 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:31.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.034+0000 7f915584a640 1 -- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91500cdfe0 msgr2=0x7f91501f1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.034+0000 7f915584a640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91500cdfe0 0x7f91501f1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.035+0000 7f915584a640 1 -- 172.21.15.133:0/1105630831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91500712b0 msgr2=0x7f91501f0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.035+0000 7f915584a640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91500712b0 0x7f91501f0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.035+0000 7f915584a640 1 -- 172.21.15.133:0/1105630831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91501e4190 con 0x7f91500ce3c0 2024-02-21T23:25:31.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.037+0000 7f915584a640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91500ce3c0 0x7f91501eee40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f914800caa0 tx=0x7f914800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:31.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.039+0000 7f9145ffb640 1 -- 172.21.15.133:0/1105630831 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148017070 con 0x7f91500ce3c0 2024-02-21T23:25:31.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.039+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91501e3440 con 0x7f91500ce3c0 2024-02-21T23:25:31.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.040+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91501e3bd0 con 0x7f91500ce3c0 2024-02-21T23:25:31.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.040+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915006a130 con 0x7f91500ce3c0 2024-02-21T23:25:31.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.042+0000 7f9145ffb640 1 -- 172.21.15.133:0/1105630831 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914800ac00 con 0x7f91500ce3c0 2024-02-21T23:25:31.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.042+0000 7f9145ffb640 1 -- 172.21.15.133:0/1105630831 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148021060 con 0x7f91500ce3c0 2024-02-21T23:25:31.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.300+0000 7f9145ffb640 1 -- 172.21.15.133:0/1105630831 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f914801a070 con 0x7f91500ce3c0 2024-02-21T23:25:31.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.300+0000 7f9145ffb640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f913406b500 0x7f913406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.301+0000 7f9155049640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f913406b500 0x7f913406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.301+0000 7f9155049640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f913406b500 0x7f913406d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9140009b40 tx=0x7f9140009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:31.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.306+0000 7f9145ffb640 1 -- 172.21.15.133:0/1105630831 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f914801de30 con 0x7f91500ce3c0 2024-02-21T23:25:31.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.316+0000 7f9145ffb640 1 -- 172.21.15.133:0/1105630831 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f9148058af0 con 0x7f91500ce3c0 2024-02-21T23:25:31.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.490+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f915014d3c0 con 0x7f91500ce3c0 2024-02-21T23:25:31.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.630+0000 7f9145ffb640 1 -- 172.21.15.133:0/1105630831 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f914805cb90 con 0x7f91500ce3c0 2024-02-21T23:25:31.617 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:31.617 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:31.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f913406b500 msgr2=0x7f913406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f913406b500 0x7f913406d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9140009b40 tx=0x7f9140009190 comp rx=0 tx=0).stop 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91500ce3c0 msgr2=0x7f91501eee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91500ce3c0 0x7f91501eee40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f914800caa0 tx=0x7f914800cf60 comp rx=0 tx=0).stop 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 shutdown_connections 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f913406b500 0x7f913406d9c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91500ce3c0 0x7f91501eee40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91500cdfe0 0x7f91501f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 --2- 172.21.15.133:0/1105630831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91500712b0 0x7f91501f0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 >> 172.21.15.133:0/1105630831 conn(0x7f915006aec0 msgr2=0x7f9150156320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.631+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 shutdown_connections 2024-02-21T23:25:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.632+0000 7f91572ad640 1 -- 172.21.15.133:0/1105630831 wait complete. 2024-02-21T23:25:31.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-21T23:25:02.902698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-21T23:25:13.993803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-21T23:25:15.604077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-21T23:25:19.468744+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-21T23:25:31.630 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-21T23:25:31.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.746+0000 7f240399c640 1 Processor -- start 2024-02-21T23:25:31.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.746+0000 7f240399c640 1 -- start start 2024-02-21T23:25:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.747+0000 7f240399c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc154240 0x7f23fc0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.747+0000 7f240399c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23fc0703c0 0x7f23fc0c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.747+0000 7f240399c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23fc0cd300 con 0x7f23fc070800 2024-02-21T23:25:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.747+0000 7f240399c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23fc0cd570 con 0x7f23fc0703c0 2024-02-21T23:25:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.747+0000 7f240399c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc15a130 con 0x7f23fc154240 2024-02-21T23:25:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.748+0000 7f2401f39640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f23fc070800 0x7f23fc15d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:54814/0 (socket says 172.21.15.133:54814) 2024-02-21T23:25:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.748+0000 7f2401f39640 1 -- 172.21.15.133:0/2022440104 learned_addr learned my addr 172.21.15.133:0/2022440104 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.748+0000 7f2401738640 1 --2- 172.21.15.133:0/2022440104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc154240 0x7f23fc0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.748+0000 7f2400f37640 1 --2- 172.21.15.133:0/2022440104 >> v2:172.21.15.133:3300/0 conn(0x7f23fc0703c0 0x7f23fc0c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.750+0000 7f2400f37640 1 -- 172.21.15.133:0/2022440104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc154240 msgr2=0x7f23fc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.750+0000 7f2400f37640 1 --2- 172.21.15.133:0/2022440104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc154240 0x7f23fc0ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.750+0000 7f2400f37640 1 -- 172.21.15.133:0/2022440104 >> v1:172.21.15.142:6789/0 conn(0x7f23fc070800 legacy=0x7f23fc15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.750+0000 7f2400f37640 1 -- 172.21.15.133:0/2022440104 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc15a410 con 0x7f23fc0703c0 2024-02-21T23:25:31.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.752+0000 7f2400f37640 1 --2- 172.21.15.133:0/2022440104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc0703c0 0x7f23fc0c90b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0097a0 tx=0x7f23ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da05cef57e98e397 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.756+0000 7f23f3fff640 1 -- 172.21.15.133:0/2022440104 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec035020 con 0x7f23fc0703c0 2024-02-21T23:25:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.756+0000 7f23f3fff640 1 -- 172.21.15.133:0/2022440104 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec031830 con 0x7f23fc0703c0 2024-02-21T23:25:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.756+0000 7f23f3fff640 1 -- 172.21.15.133:0/2022440104 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec031c00 con 0x7f23fc0703c0 2024-02-21T23:25:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.757+0000 7f240399c640 1 -- 172.21.15.133:0/2022440104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc0703c0 msgr2=0x7f23fc0c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.757+0000 7f240399c640 1 --2- 172.21.15.133:0/2022440104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc0703c0 0x7f23fc0c90b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0097a0 tx=0x7f23ec02b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.757+0000 7f240399c640 1 -- 172.21.15.133:0/2022440104 shutdown_connections 2024-02-21T23:25:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.757+0000 7f240399c640 1 --2- 172.21.15.133:0/2022440104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc0703c0 0x7f23fc0c90b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.757+0000 7f240399c640 1 --2- 172.21.15.133:0/2022440104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc154240 0x7f23fc0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.757+0000 7f240399c640 1 -- 172.21.15.133:0/2022440104 >> 172.21.15.133:0/2022440104 conn(0x7f23fc061b50 msgr2=0x7f23fc072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.757+0000 7f240399c640 1 -- 172.21.15.133:0/2022440104 shutdown_connections 2024-02-21T23:25:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.757+0000 7f240399c640 1 -- 172.21.15.133:0/2022440104 wait complete. 2024-02-21T23:25:31.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.758+0000 7f240399c640 1 Processor -- start 2024-02-21T23:25:31.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.758+0000 7f240399c640 1 -- start start 2024-02-21T23:25:31.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f240399c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23fc0703c0 0x7f23fc14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f240399c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc070800 0x7f23fc14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f240399c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc154240 0x7f23fc1f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f240399c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc154620 con 0x7f23fc154240 2024-02-21T23:25:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f240399c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc15a9d0 con 0x7f23fc0703c0 2024-02-21T23:25:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f240399c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc15b280 con 0x7f23fc070800 2024-02-21T23:25:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f2401f39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc154240 0x7f23fc1f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f2400f37640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc070800 0x7f23fc14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f2401738640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23fc0703c0 0x7f23fc14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f2400f37640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc070800 0x7f23fc14ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:33934/0 (socket says 172.21.15.133:33934) 2024-02-21T23:25:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.759+0000 7f2400f37640 1 -- 172.21.15.133:0/3873622779 learned_addr learned my addr 172.21.15.133:0/3873622779 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:31.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.762+0000 7f2400f37640 1 -- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23fc0703c0 msgr2=0x7f23fc14aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.762+0000 7f2400f37640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23fc0703c0 0x7f23fc14aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.762+0000 7f2400f37640 1 -- 172.21.15.133:0/3873622779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc154240 msgr2=0x7f23fc1f0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:31.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.762+0000 7f2400f37640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc154240 0x7f23fc1f0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:31.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.762+0000 7f2400f37640 1 -- 172.21.15.133:0/3873622779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc14b430 con 0x7f23fc070800 2024-02-21T23:25:31.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.765+0000 7f2400f37640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc070800 0x7f23fc14ca90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0098c0 tx=0x7f23ec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.767+0000 7f23f1ffb640 1 -- 172.21.15.133:0/3873622779 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec040070 con 0x7f23fc070800 2024-02-21T23:25:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.767+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc1f00b0 con 0x7f23fc070800 2024-02-21T23:25:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.767+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc15b520 con 0x7f23fc070800 2024-02-21T23:25:31.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.768+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23fc06a130 con 0x7f23fc070800 2024-02-21T23:25:31.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.770+0000 7f23f1ffb640 1 -- 172.21.15.133:0/3873622779 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec03e070 con 0x7f23fc070800 2024-02-21T23:25:31.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.770+0000 7f23f1ffb640 1 -- 172.21.15.133:0/3873622779 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec035040 con 0x7f23fc070800 2024-02-21T23:25:31.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.794+0000 7f23f1ffb640 1 -- 172.21.15.133:0/3873622779 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23ec038020 con 0x7f23fc070800 2024-02-21T23:25:31.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.795+0000 7f23f1ffb640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f23e006b4d0 0x7f23e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:31.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.795+0000 7f2401738640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f23e006b4d0 0x7f23e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:31.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:31.795+0000 7f2401738640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f23e006b4d0 0x7f23e006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23f80113e0 tx=0x7f23f800b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:32.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.046+0000 7f23f1ffb640 1 -- 172.21.15.133:0/3873622779 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f23ec078480 con 0x7f23fc070800 2024-02-21T23:25:32.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.059+0000 7f23f1ffb640 1 -- 172.21.15.133:0/3873622779 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f23ec052020 con 0x7f23fc070800 2024-02-21T23:25:32.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.234+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23fc06b7b0 con 0x7f23fc070800 2024-02-21T23:25:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.257+0000 7f23f1ffb640 1 -- 172.21.15.133:0/3873622779 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23ec031800 con 0x7f23fc070800 2024-02-21T23:25:32.244 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f23e006b4d0 msgr2=0x7f23e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f23e006b4d0 0x7f23e006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23f80113e0 tx=0x7f23f800b040 comp rx=0 tx=0).stop 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc070800 msgr2=0x7f23fc14ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc070800 0x7f23fc14ca90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0098c0 tx=0x7f23ec0046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 shutdown_connections 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f23e006b4d0 0x7f23e006d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc154240 0x7f23fc1f0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23fc070800 0x7f23fc14ca90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 --2- 172.21.15.133:0/3873622779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23fc0703c0 0x7f23fc14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 >> 172.21.15.133:0/3873622779 conn(0x7f23fc061b50 msgr2=0x7f23fc151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 shutdown_connections 2024-02-21T23:25:32.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.259+0000 7f240399c640 1 -- 172.21.15.133:0/3873622779 wait complete. 2024-02-21T23:25:32.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-21T23:25:32.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.373+0000 7f85bb71e640 1 Processor -- start 2024-02-21T23:25:32.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.373+0000 7f85bb71e640 1 -- start start 2024-02-21T23:25:32.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.374+0000 7f85bb71e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f85b4154450 0x7f85b406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.374+0000 7f85bb71e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b415a720 con 0x7f85b415a290 2024-02-21T23:25:32.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.374+0000 7f85bb71e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b415a9e0 con 0x7f85b406ff00 2024-02-21T23:25:32.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.374+0000 7f85bb71e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b406c2f0 con 0x7f85b4154450 2024-02-21T23:25:32.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.375+0000 7f85b8cb9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f85b415a290 0x7f85b40c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44832/0 (socket says 172.21.15.133:44832) 2024-02-21T23:25:32.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.375+0000 7f85b8cb9640 1 -- 172.21.15.133:0/2159512269 learned_addr learned my addr 172.21.15.133:0/2159512269 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:32.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.375+0000 7f85b94ba640 1 --2- 172.21.15.133:0/2159512269 >> v2:172.21.15.142:3300/0 conn(0x7f85b4154450 0x7f85b406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.377+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662229170 0 0) 0x7f85b415a720 con 0x7f85b415a290 2024-02-21T23:25:32.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.377+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a0003520 con 0x7f85b415a290 2024-02-21T23:25:32.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3419833155 0 0) 0x7f85a0003520 con 0x7f85b415a290 2024-02-21T23:25:32.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 >> v2:172.21.15.142:3300/0 conn(0x7f85b4154450 msgr2=0x7f85b406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:32.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 --2- 172.21.15.133:0/2159512269 >> v2:172.21.15.142:3300/0 conn(0x7f85b4154450 0x7f85b406f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 >> v1:172.21.15.142:6789/0 conn(0x7f85b406ff00 legacy=0x7f85b40c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b415acc0 con 0x7f85b415a290 2024-02-21T23:25:32.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 797941756 0 0) 0x7f85a4001670 con 0x7f85b415a290 2024-02-21T23:25:32.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 >> v1:172.21.15.133:6789/0 conn(0x7f85b415a290 legacy=0x7f85b40c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0003a00 0x7f85a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85a0004800 0x7f85a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85a00051c0 0x7f85a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b406c2f0 con 0x7f85a0003a00 2024-02-21T23:25:32.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a000c0a0 con 0x7f85a00051c0 2024-02-21T23:25:32.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a000c2a0 con 0x7f85a0004800 2024-02-21T23:25:32.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.381+0000 7f85b8cb9640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85a0004800 0x7f85a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.383+0000 7f85b94ba640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0003a00 0x7f85a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.384+0000 7f85b9cbb640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85a00051c0 0x7f85a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.384+0000 7f85b8cb9640 1 -- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85a00051c0 msgr2=0x7f85a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.384+0000 7f85b8cb9640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85a00051c0 0x7f85a000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.384+0000 7f85b8cb9640 1 -- 172.21.15.133:0/2159512269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0003a00 msgr2=0x7f85a0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.384+0000 7f85b8cb9640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0003a00 0x7f85a0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.384+0000 7f85b9cbb640 1 -- 172.21.15.133:0/2159512269 reap_dead start 2024-02-21T23:25:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.384+0000 7f85b8cb9640 1 -- 172.21.15.133:0/2159512269 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0003520 con 0x7f85a0004800 2024-02-21T23:25:32.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.386+0000 7f85b8cb9640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85a0004800 0x7f85a0004c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c110 tx=0x7f85a0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63d7d32f4d75c73e server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:32.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4007c40 con 0x7f85a0004800 2024-02-21T23:25:32.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a4030070 con 0x7f85a0004800 2024-02-21T23:25:32.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85abfff640 1 -- 172.21.15.133:0/2159512269 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4005d70 con 0x7f85a0004800 2024-02-21T23:25:32.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85bb71e640 1 -- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85a0004800 msgr2=0x7f85a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85bb71e640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85a0004800 0x7f85a0004c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c110 tx=0x7f85a0005830 comp rx=0 tx=0).stop 2024-02-21T23:25:32.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85bb71e640 1 -- 172.21.15.133:0/2159512269 shutdown_connections 2024-02-21T23:25:32.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85bb71e640 1 --2- 172.21.15.133:0/2159512269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85a0004800 0x7f85a0004c80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85bb71e640 1 -- 172.21.15.133:0/2159512269 >> 172.21.15.133:0/2159512269 conn(0x7f85b4070ec0 msgr2=0x7f85b40712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:32.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85bb71e640 1 -- 172.21.15.133:0/2159512269 shutdown_connections 2024-02-21T23:25:32.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.390+0000 7f85bb71e640 1 -- 172.21.15.133:0/2159512269 wait complete. 2024-02-21T23:25:32.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.392+0000 7f85bb71e640 1 Processor -- start 2024-02-21T23:25:32.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.392+0000 7f85bb71e640 1 -- start start 2024-02-21T23:25:32.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.392+0000 7f85bb71e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85a0004800 0x7f85b41502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.392+0000 7f85bb71e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85b4155720 0x7f85b4158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.392+0000 7f85bb71e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b4151440 0x7f85b41f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.392+0000 7f85bb71e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b4154830 con 0x7f85b4151440 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.392+0000 7f85bb71e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b406f4a0 con 0x7f85a0004800 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.392+0000 7f85bb71e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b41549b0 con 0x7f85b4155720 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.393+0000 7f85b8cb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85b4155720 0x7f85b4158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.393+0000 7f85b9cbb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b4151440 0x7f85b41f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.393+0000 7f85b94ba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85a0004800 0x7f85b41502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.393+0000 7f85b8cb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85b4155720 0x7f85b4158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:33952/0 (socket says 172.21.15.133:33952) 2024-02-21T23:25:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.393+0000 7f85b8cb9640 1 -- 172.21.15.133:0/1100875302 learned_addr learned my addr 172.21.15.133:0/1100875302 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.396+0000 7f85b8cb9640 1 -- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85a0004800 msgr2=0x7f85b41502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.396+0000 7f85b8cb9640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85a0004800 0x7f85b41502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.396+0000 7f85b8cb9640 1 -- 172.21.15.133:0/1100875302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b4151440 msgr2=0x7f85b41f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.396+0000 7f85b8cb9640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b4151440 0x7f85b41f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.396+0000 7f85b8cb9640 1 -- 172.21.15.133:0/1100875302 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b414f3b0 con 0x7f85b4155720 2024-02-21T23:25:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.398+0000 7f85b8cb9640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85b4155720 0x7f85b4158ac0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c200 tx=0x7f85a402e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:32.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.402+0000 7f85a9ffb640 1 -- 172.21.15.133:0/1100875302 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4004bf0 con 0x7f85b4155720 2024-02-21T23:25:32.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.402+0000 7f85a9ffb640 1 -- 172.21.15.133:0/1100875302 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a4000ed0 con 0x7f85b4155720 2024-02-21T23:25:32.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.402+0000 7f85a9ffb640 1 -- 172.21.15.133:0/1100875302 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4007070 con 0x7f85b4155720 2024-02-21T23:25:32.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.402+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b4155c60 con 0x7f85b4155720 2024-02-21T23:25:32.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.402+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b414e5f0 con 0x7f85b4155720 2024-02-21T23:25:32.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.403+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b406a130 con 0x7f85b4155720 2024-02-21T23:25:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.421+0000 7f85a9ffb640 1 -- 172.21.15.133:0/1100875302 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85a4007210 con 0x7f85b4155720 2024-02-21T23:25:32.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.422+0000 7f85a9ffb640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f859806b520 0x7f859806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.422+0000 7f85b94ba640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f859806b520 0x7f859806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.423+0000 7f85b94ba640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f859806b520 0x7f859806d9e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009fd0 tx=0x7f85b000b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:32.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.424+0000 7f85a9ffb640 1 -- 172.21.15.133:0/1100875302 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f85a4007cc0 con 0x7f85b4155720 2024-02-21T23:25:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.672+0000 7f85a9ffb640 1 -- 172.21.15.133:0/1100875302 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f85a407a1f0 con 0x7f85b4155720 2024-02-21T23:25:32.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.844+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85b40c8db0 con 0x7f85b4155720 2024-02-21T23:25:32.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.850+0000 7f85a9ffb640 1 -- 172.21.15.133:0/1100875302 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85a4080070 con 0x7f85b4155720 2024-02-21T23:25:32.836 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f859806b520 msgr2=0x7f859806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f859806b520 0x7f859806d9e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009fd0 tx=0x7f85b000b040 comp rx=0 tx=0).stop 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85b4155720 msgr2=0x7f85b4158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85b4155720 0x7f85b4158ac0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c200 tx=0x7f85a402e070 comp rx=0 tx=0).stop 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 shutdown_connections 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f859806b520 0x7f859806d9e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b4151440 0x7f85b41f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85b4155720 0x7f85b4158ac0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 --2- 172.21.15.133:0/1100875302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85a0004800 0x7f85b41502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 >> 172.21.15.133:0/1100875302 conn(0x7f85b4070ec0 msgr2=0x7f85b40af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 shutdown_connections 2024-02-21T23:25:32.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.852+0000 7f85bb71e640 1 -- 172.21.15.133:0/1100875302 wait complete. 2024-02-21T23:25:32.849 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-21T23:25:32.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.967+0000 7fa0c09aa640 1 Processor -- start 2024-02-21T23:25:32.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.967+0000 7fa0c09aa640 1 -- start start 2024-02-21T23:25:32.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.967+0000 7fa0c09aa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa0bc0cfe10 0x7fa0bc0d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.967+0000 7fa0c09aa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 0x7fa0bc0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.967+0000 7fa0c09aa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa0bc0ca980 0x7fa0bc0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.967+0000 7fa0c09aa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc0cc400 con 0x7fa0bc0cfe10 2024-02-21T23:25:32.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.967+0000 7fa0c09aa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc0cc100 con 0x7fa0bc0ca980 2024-02-21T23:25:32.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.967+0000 7fa0c09aa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15a6b0 con 0x7fa0bc15a170 2024-02-21T23:25:32.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.968+0000 7fa0bad9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa0bc0ca980 0x7fa0bc0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.968+0000 7fa0b9d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 0x7fa0bc0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.968+0000 7fa0ba59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa0bc0cfe10 0x7fa0bc0d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.968+0000 7fa0bad9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa0bc0ca980 0x7fa0bc0cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:46804/0 (socket says 172.21.15.133:46804) 2024-02-21T23:25:32.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.968+0000 7fa0bad9d640 1 -- 172.21.15.133:0/2325266567 learned_addr learned my addr 172.21.15.133:0/2325266567 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.970+0000 7fa0bad9d640 1 -- 172.21.15.133:0/2325266567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 msgr2=0x7fa0bc0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.970+0000 7fa0bad9d640 1 --2- 172.21.15.133:0/2325266567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 0x7fa0bc0ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.970+0000 7fa0bad9d640 1 -- 172.21.15.133:0/2325266567 >> v2:172.21.15.133:3300/0 conn(0x7fa0bc0cfe10 msgr2=0x7fa0bc0d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.970+0000 7fa0bad9d640 1 --2- 172.21.15.133:0/2325266567 >> v2:172.21.15.133:3300/0 conn(0x7fa0bc0cfe10 0x7fa0bc0d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.970+0000 7fa0bad9d640 1 -- 172.21.15.133:0/2325266567 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc15a970 con 0x7fa0bc0ca980 2024-02-21T23:25:32.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.972+0000 7fa0bad9d640 1 --2- 172.21.15.133:0/2325266567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0ca980 0x7fa0bc0cc890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0d0a00 tx=0x7fa0b402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17e22f9b25beaf9a server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:32.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.976+0000 7fa0b959a640 1 -- 172.21.15.133:0/2325266567 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b402c9e0 con 0x7fa0bc0ca980 2024-02-21T23:25:32.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.976+0000 7fa0b959a640 1 -- 172.21.15.133:0/2325266567 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b402cb80 con 0x7fa0bc0ca980 2024-02-21T23:25:32.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.976+0000 7fa0b959a640 1 -- 172.21.15.133:0/2325266567 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40343f0 con 0x7fa0bc0ca980 2024-02-21T23:25:32.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.977+0000 7fa0c09aa640 1 -- 172.21.15.133:0/2325266567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0ca980 msgr2=0x7fa0bc0cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.977+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/2325266567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0ca980 0x7fa0bc0cc890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0d0a00 tx=0x7fa0b402be40 comp rx=0 tx=0).stop 2024-02-21T23:25:32.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.977+0000 7fa0c09aa640 1 -- 172.21.15.133:0/2325266567 shutdown_connections 2024-02-21T23:25:32.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.977+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/2325266567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0ca980 0x7fa0bc0cc890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.977+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/2325266567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 0x7fa0bc0ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.977+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/2325266567 >> v2:172.21.15.133:3300/0 conn(0x7fa0bc0cfe10 0x7fa0bc0d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.977+0000 7fa0c09aa640 1 -- 172.21.15.133:0/2325266567 >> 172.21.15.133:0/2325266567 conn(0x7fa0bc061b50 msgr2=0x7fa0bc061f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:32.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.979+0000 7fa0c09aa640 1 -- 172.21.15.133:0/2325266567 shutdown_connections 2024-02-21T23:25:32.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.979+0000 7fa0c09aa640 1 -- 172.21.15.133:0/2325266567 wait complete. 2024-02-21T23:25:32.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.980+0000 7fa0c09aa640 1 Processor -- start 2024-02-21T23:25:32.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.980+0000 7fa0c09aa640 1 -- start start 2024-02-21T23:25:32.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.981+0000 7fa0c09aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0bc0ca980 0x7fa0bc155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.981+0000 7fa0c09aa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0cfe10 0x7fa0bc0c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.981+0000 7fa0c09aa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 0x7fa0bc151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:32.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.981+0000 7fa0c09aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0ce1a0 con 0x7fa0bc0ca980 2024-02-21T23:25:32.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.981+0000 7fa0c09aa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15af30 con 0x7fa0bc0cfe10 2024-02-21T23:25:32.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.981+0000 7fa0c09aa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0ce320 con 0x7fa0bc15a170 2024-02-21T23:25:32.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.981+0000 7fa0b9d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0cfe10 0x7fa0bc0c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.982+0000 7fa0bad9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 0x7fa0bc151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.982+0000 7fa0ba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0bc0ca980 0x7fa0bc155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:32.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.982+0000 7fa0b9d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0cfe10 0x7fa0bc0c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:46806/0 (socket says 172.21.15.133:46806) 2024-02-21T23:25:32.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.982+0000 7fa0b9d9b640 1 -- 172.21.15.133:0/3178511925 learned_addr learned my addr 172.21.15.133:0/3178511925 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:32.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.985+0000 7fa0b9d9b640 1 -- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 msgr2=0x7fa0bc151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.985+0000 7fa0b9d9b640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 0x7fa0bc151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.985+0000 7fa0b9d9b640 1 -- 172.21.15.133:0/3178511925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0bc0ca980 msgr2=0x7fa0bc155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:32.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.985+0000 7fa0b9d9b640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0bc0ca980 0x7fa0bc155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:32.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.985+0000 7fa0b9d9b640 1 -- 172.21.15.133:0/3178511925 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc155a40 con 0x7fa0bc0cfe10 2024-02-21T23:25:32.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.987+0000 7fa0b9d9b640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0cfe10 0x7fa0bc0c9040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00e870 tx=0x7fa0ac00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:32.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.989+0000 7fa0aaffd640 1 -- 172.21.15.133:0/3178511925 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac017070 con 0x7fa0bc0cfe10 2024-02-21T23:25:32.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.989+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc1f0110 con 0x7fa0bc0cfe10 2024-02-21T23:25:32.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.989+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc1f06d0 con 0x7fa0bc0cfe10 2024-02-21T23:25:32.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.990+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc062460 con 0x7fa0bc0cfe10 2024-02-21T23:25:32.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.990+0000 7fa0aaffd640 1 -- 172.21.15.133:0/3178511925 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac005990 con 0x7fa0bc0cfe10 2024-02-21T23:25:32.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:32.990+0000 7fa0aaffd640 1 -- 172.21.15.133:0/3178511925 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac021060 con 0x7fa0bc0cfe10 2024-02-21T23:25:33.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.241+0000 7fa0aaffd640 1 -- 172.21.15.133:0/3178511925 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0ac01a070 con 0x7fa0bc0cfe10 2024-02-21T23:25:33.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.241+0000 7fa0aaffd640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa09c06b4d0 0x7fa09c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:33.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.242+0000 7fa0ba59c640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa09c06b4d0 0x7fa09c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:33.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.242+0000 7fa0ba59c640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa09c06b4d0 0x7fa09c06d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009b40 tx=0x7fa0b0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:33.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.244+0000 7fa0aaffd640 1 -- 172.21.15.133:0/3178511925 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa0ac056c10 con 0x7fa0bc0cfe10 2024-02-21T23:25:33.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.255+0000 7fa0aaffd640 1 -- 172.21.15.133:0/3178511925 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fa0bc062460 con 0x7fa0bc0cfe10 2024-02-21T23:25:33.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.434+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0bc06a130 con 0x7fa0bc0cfe10 2024-02-21T23:25:33.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.457+0000 7fa0aaffd640 1 -- 172.21.15.133:0/3178511925 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0ac058b70 con 0x7fa0bc0cfe10 2024-02-21T23:25:33.444 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa09c06b4d0 msgr2=0x7fa09c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa09c06b4d0 0x7fa09c06d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009b40 tx=0x7fa0b0009190 comp rx=0 tx=0).stop 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0cfe10 msgr2=0x7fa0bc0c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0cfe10 0x7fa0bc0c9040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00e870 tx=0x7fa0ac00ed30 comp rx=0 tx=0).stop 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 shutdown_connections 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa09c06b4d0 0x7fa09c06d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0bc15a170 0x7fa0bc151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0bc0cfe10 0x7fa0bc0c9040 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 --2- 172.21.15.133:0/3178511925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0bc0ca980 0x7fa0bc155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 >> 172.21.15.133:0/3178511925 conn(0x7fa0bc061b50 msgr2=0x7fa0bc158850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:33.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 shutdown_connections 2024-02-21T23:25:33.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.460+0000 7fa0c09aa640 1 -- 172.21.15.133:0/3178511925 wait complete. 2024-02-21T23:25:33.458 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-21T23:25:33.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.575+0000 7fe201ed1640 1 Processor -- start 2024-02-21T23:25:33.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.575+0000 7fe201ed1640 1 -- start start 2024-02-21T23:25:33.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.576+0000 7fe201ed1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1fc154240 0x7fe1fc0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:33.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.576+0000 7fe201ed1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1fc15d630 0x7fe1fc15da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.576+0000 7fe201ed1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1fc0c8990 con 0x7fe1fc0c8420 2024-02-21T23:25:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.576+0000 7fe201ed1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c3ff0 con 0x7fe1fc15d630 2024-02-21T23:25:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.576+0000 7fe201ed1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c8c50 con 0x7fe1fc154240 2024-02-21T23:25:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.576+0000 7fe1fbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1fc15d630 0x7fe1fc15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.577+0000 7fe1fbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1fc15d630 0x7fe1fc15da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50610/0 (socket says 172.21.15.133:50610) 2024-02-21T23:25:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.577+0000 7fe1fbfff640 1 -- 172.21.15.133:0/958083641 learned_addr learned my addr 172.21.15.133:0/958083641 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:33.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.577+0000 7fe1fb7fe640 1 --2- 172.21.15.133:0/958083641 >> v2:172.21.15.142:3300/0 conn(0x7fe1fc154240 0x7fe1fc0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:33.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.579+0000 7fe1fbfff640 1 -- 172.21.15.133:0/958083641 >> v2:172.21.15.142:3300/0 conn(0x7fe1fc154240 msgr2=0x7fe1fc0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:33.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.579+0000 7fe1fbfff640 1 --2- 172.21.15.133:0/958083641 >> v2:172.21.15.142:3300/0 conn(0x7fe1fc154240 0x7fe1fc0c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.579+0000 7fe1fbfff640 1 -- 172.21.15.133:0/958083641 >> v1:172.21.15.142:6789/0 conn(0x7fe1fc0c8420 legacy=0x7fe1fc0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:33.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.579+0000 7fe1fbfff640 1 -- 172.21.15.133:0/958083641 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0c39a0 con 0x7fe1fc15d630 2024-02-21T23:25:33.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.581+0000 7fe1fbfff640 1 --2- 172.21.15.133:0/958083641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc15d630 0x7fe1fc15da40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0c4170 tx=0x7fe1f402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df480539658164a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.584+0000 7fe1fa7fc640 1 -- 172.21.15.133:0/958083641 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f402cb90 con 0x7fe1fc15d630 2024-02-21T23:25:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.584+0000 7fe1fa7fc640 1 -- 172.21.15.133:0/958083641 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f402cd30 con 0x7fe1fc15d630 2024-02-21T23:25:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.584+0000 7fe1fa7fc640 1 -- 172.21.15.133:0/958083641 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4034610 con 0x7fe1fc15d630 2024-02-21T23:25:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.585+0000 7fe201ed1640 1 -- 172.21.15.133:0/958083641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc15d630 msgr2=0x7fe1fc15da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.585+0000 7fe201ed1640 1 --2- 172.21.15.133:0/958083641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc15d630 0x7fe1fc15da40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0c4170 tx=0x7fe1f402bff0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.585+0000 7fe201ed1640 1 -- 172.21.15.133:0/958083641 shutdown_connections 2024-02-21T23:25:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.585+0000 7fe201ed1640 1 --2- 172.21.15.133:0/958083641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc15d630 0x7fe1fc15da40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.585+0000 7fe201ed1640 1 --2- 172.21.15.133:0/958083641 >> v2:172.21.15.142:3300/0 conn(0x7fe1fc154240 0x7fe1fc0c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.585+0000 7fe201ed1640 1 -- 172.21.15.133:0/958083641 >> 172.21.15.133:0/958083641 conn(0x7fe1fc06f7f0 msgr2=0x7fe1fc061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:33.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.585+0000 7fe201ed1640 1 -- 172.21.15.133:0/958083641 shutdown_connections 2024-02-21T23:25:33.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.585+0000 7fe201ed1640 1 -- 172.21.15.133:0/958083641 wait complete. 2024-02-21T23:25:33.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.586+0000 7fe201ed1640 1 Processor -- start 2024-02-21T23:25:33.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.587+0000 7fe201ed1640 1 -- start start 2024-02-21T23:25:33.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.587+0000 7fe201ed1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc0c8420 0x7fe1fc15a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:33.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.587+0000 7fe201ed1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1fc154240 0x7fe1fc1f0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:33.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.587+0000 7fe201ed1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1fc15d630 0x7fe1fc1f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:33.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.587+0000 7fe201ed1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc15df80 con 0x7fe1fc0c8420 2024-02-21T23:25:33.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.587+0000 7fe201ed1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0ca130 con 0x7fe1fc154240 2024-02-21T23:25:33.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.587+0000 7fe201ed1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c36a0 con 0x7fe1fc15d630 2024-02-21T23:25:33.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.588+0000 7fe1fb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc0c8420 0x7fe1fc15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:33.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.588+0000 7fe1faffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1fc154240 0x7fe1fc1f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:33.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.588+0000 7fe1fbfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1fc15d630 0x7fe1fc1f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:33.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.588+0000 7fe1fb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc0c8420 0x7fe1fc15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50624/0 (socket says 172.21.15.133:50624) 2024-02-21T23:25:33.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.588+0000 7fe1fb7fe640 1 -- 172.21.15.133:0/3584410140 learned_addr learned my addr 172.21.15.133:0/3584410140 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:33.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.591+0000 7fe1fb7fe640 1 -- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1fc15d630 msgr2=0x7fe1fc1f08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:33.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.591+0000 7fe1fb7fe640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1fc15d630 0x7fe1fc1f08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.591+0000 7fe1fb7fe640 1 -- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1fc154240 msgr2=0x7fe1fc1f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:33.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.591+0000 7fe1fb7fe640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1fc154240 0x7fe1fc1f0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:33.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.591+0000 7fe1fb7fe640 1 -- 172.21.15.133:0/3584410140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc1efe60 con 0x7fe1fc0c8420 2024-02-21T23:25:33.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.594+0000 7fe1fb7fe640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc0c8420 0x7fe1fc15a270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00e9b0 tx=0x7fe1ec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:33.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.596+0000 7fe1e3fff640 1 -- 172.21.15.133:0/3584410140 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec017070 con 0x7fe1fc0c8420 2024-02-21T23:25:33.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.596+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1fc1f3570 con 0x7fe1fc0c8420 2024-02-21T23:25:33.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.596+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc1f3b30 con 0x7fe1fc0c8420 2024-02-21T23:25:33.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.597+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1fc06a130 con 0x7fe1fc0c8420 2024-02-21T23:25:33.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.597+0000 7fe1e3fff640 1 -- 172.21.15.133:0/3584410140 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec00c9a0 con 0x7fe1fc0c8420 2024-02-21T23:25:33.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.597+0000 7fe1e3fff640 1 -- 172.21.15.133:0/3584410140 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec021060 con 0x7fe1fc0c8420 2024-02-21T23:25:33.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.843+0000 7fe1e3fff640 1 -- 172.21.15.133:0/3584410140 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1ec01a070 con 0x7fe1fc0c8420 2024-02-21T23:25:33.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.844+0000 7fe1e3fff640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1e806b4d0 0x7fe1e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:33.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.844+0000 7fe1faffd640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1e806b4d0 0x7fe1e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:33.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.845+0000 7fe1faffd640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1e806b4d0 0x7fe1e806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc1f0f90 tx=0x7fe1f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.861+0000 7fe1e3fff640 1 -- 172.21.15.133:0/3584410140 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe1ec057260 con 0x7fe1fc0c8420 2024-02-21T23:25:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:33.861+0000 7fe1e3fff640 1 -- 172.21.15.133:0/3584410140 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fe1ec0591c0 con 0x7fe1fc0c8420 2024-02-21T23:25:34.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.035+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe1fc062760 con 0x7fe1fc0c8420 2024-02-21T23:25:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.040+0000 7fe1e3fff640 1 -- 172.21.15.133:0/3584410140 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe1ec05d260 con 0x7fe1fc0c8420 2024-02-21T23:25:34.027 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:25:34.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1e806b4d0 msgr2=0x7fe1e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1e806b4d0 0x7fe1e806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc1f0f90 tx=0x7fe1f0009040 comp rx=0 tx=0).stop 2024-02-21T23:25:34.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc0c8420 msgr2=0x7fe1fc15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc0c8420 0x7fe1fc15a270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00e9b0 tx=0x7fe1ec00ee70 comp rx=0 tx=0).stop 2024-02-21T23:25:34.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 shutdown_connections 2024-02-21T23:25:34.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1e806b4d0 0x7fe1e806d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1fc15d630 0x7fe1fc1f08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1fc154240 0x7fe1fc1f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 --2- 172.21.15.133:0/3584410140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1fc0c8420 0x7fe1fc15a270 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 >> 172.21.15.133:0/3584410140 conn(0x7fe1fc06f7f0 msgr2=0x7fe1fc150ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:34.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 shutdown_connections 2024-02-21T23:25:34.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.042+0000 7fe201ed1640 1 -- 172.21.15.133:0/3584410140 wait complete. 2024-02-21T23:25:34.040 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-21T23:25:34.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.155+0000 7fa466a1c640 1 Processor -- start 2024-02-21T23:25:34.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.155+0000 7fa466a1c640 1 -- start start 2024-02-21T23:25:34.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.156+0000 7fa466a1c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa4600c9830 0x7fa460150940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:34.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.156+0000 7fa466a1c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa460073210 con 0x7fa4600c9450 2024-02-21T23:25:34.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.156+0000 7fa466a1c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46015a280 con 0x7fa4600c7790 2024-02-21T23:25:34.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.156+0000 7fa466a1c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa46015a560 con 0x7fa4600c9830 2024-02-21T23:25:34.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.156+0000 7fa45ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa4600c9450 0x7fa460072a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44836/0 (socket says 172.21.15.133:44836) 2024-02-21T23:25:34.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.156+0000 7fa45ffff640 1 -- 172.21.15.133:0/2400733610 learned_addr learned my addr 172.21.15.133:0/2400733610 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:34.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.157+0000 7fa464fb9640 1 --2- 172.21.15.133:0/2400733610 >> v2:172.21.15.142:3300/0 conn(0x7fa4600c9830 0x7fa460150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.159+0000 7fa45effd640 1 -- 172.21.15.133:0/2400733610 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010832431 0 0) 0x7fa460073210 con 0x7fa4600c9450 2024-02-21T23:25:34.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.159+0000 7fa45effd640 1 -- 172.21.15.133:0/2400733610 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa448003520 con 0x7fa4600c9450 2024-02-21T23:25:34.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.162+0000 7fa464fb9640 1 -- 172.21.15.133:0/2400733610 >> v1:172.21.15.142:6789/0 conn(0x7fa4600c7790 legacy=0x7fa46015d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.162+0000 7fa464fb9640 1 -- 172.21.15.133:0/2400733610 >> v1:172.21.15.133:6789/0 conn(0x7fa4600c9450 legacy=0x7fa460072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.162+0000 7fa464fb9640 1 -- 172.21.15.133:0/2400733610 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46015a840 con 0x7fa4600c9830 2024-02-21T23:25:34.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.166+0000 7fa464fb9640 1 --2- 172.21.15.133:0/2400733610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4600c9830 0x7fa460150940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa458009d90 tx=0x7fa45802bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=541f1e363d75fc24 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:34.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa45effd640 1 -- 172.21.15.133:0/2400733610 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458039070 con 0x7fa4600c9830 2024-02-21T23:25:34.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa45effd640 1 -- 172.21.15.133:0/2400733610 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45802cd80 con 0x7fa4600c9830 2024-02-21T23:25:34.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa45effd640 1 -- 172.21.15.133:0/2400733610 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458034660 con 0x7fa4600c9830 2024-02-21T23:25:34.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa466a1c640 1 -- 172.21.15.133:0/2400733610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4600c9830 msgr2=0x7fa460150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa466a1c640 1 --2- 172.21.15.133:0/2400733610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4600c9830 0x7fa460150940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa458009d90 tx=0x7fa45802bfd0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa466a1c640 1 -- 172.21.15.133:0/2400733610 shutdown_connections 2024-02-21T23:25:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa466a1c640 1 --2- 172.21.15.133:0/2400733610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4600c9830 0x7fa460150940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa466a1c640 1 -- 172.21.15.133:0/2400733610 >> 172.21.15.133:0/2400733610 conn(0x7fa46006bfd0 msgr2=0x7fa46006c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa466a1c640 1 -- 172.21.15.133:0/2400733610 shutdown_connections 2024-02-21T23:25:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.171+0000 7fa466a1c640 1 -- 172.21.15.133:0/2400733610 wait complete. 2024-02-21T23:25:34.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.173+0000 7fa466a1c640 1 Processor -- start 2024-02-21T23:25:34.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.173+0000 7fa466a1c640 1 -- start start 2024-02-21T23:25:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa466a1c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4600c7790 0x7fa46015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa466a1c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4600c9450 0x7fa4600cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa466a1c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600c9830 0x7fa4601588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa466a1c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46015dff0 con 0x7fa4600c9830 2024-02-21T23:25:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa466a1c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46015de70 con 0x7fa4600c7790 2024-02-21T23:25:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa466a1c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46015cb50 con 0x7fa4600c9450 2024-02-21T23:25:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa464fb9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600c9830 0x7fa4601588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa45ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4600c7790 0x7fa46015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4600c9450 0x7fa4600cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4600c9450 0x7fa4600cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:33986/0 (socket says 172.21.15.133:33986) 2024-02-21T23:25:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.174+0000 7fa45f7fe640 1 -- 172.21.15.133:0/3386231033 learned_addr learned my addr 172.21.15.133:0/3386231033 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:34.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.178+0000 7fa464fb9640 1 -- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4600c9450 msgr2=0x7fa4600cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.178+0000 7fa464fb9640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4600c9450 0x7fa4600cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.178+0000 7fa464fb9640 1 -- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4600c7790 msgr2=0x7fa46015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.178+0000 7fa464fb9640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4600c7790 0x7fa46015bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.178+0000 7fa464fb9640 1 -- 172.21.15.133:0/3386231033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4601f3120 con 0x7fa4600c9830 2024-02-21T23:25:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.180+0000 7fa464fb9640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600c9830 0x7fa4601588e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa46006cfb0 tx=0x7fa458004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:34.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.184+0000 7fa45cff9640 1 -- 172.21.15.133:0/3386231033 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458039050 con 0x7fa4600c9830 2024-02-21T23:25:34.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.184+0000 7fa45cff9640 1 -- 172.21.15.133:0/3386231033 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458045070 con 0x7fa4600c9830 2024-02-21T23:25:34.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.184+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa460148c10 con 0x7fa4600c9830 2024-02-21T23:25:34.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.184+0000 7fa45cff9640 1 -- 172.21.15.133:0/3386231033 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458009450 con 0x7fa4600c9830 2024-02-21T23:25:34.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.184+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4601f36a0 con 0x7fa4600c9830 2024-02-21T23:25:34.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.185+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46006a130 con 0x7fa4600c9830 2024-02-21T23:25:34.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.203+0000 7fa45cff9640 1 -- 172.21.15.133:0/3386231033 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa45803e850 con 0x7fa4600c9830 2024-02-21T23:25:34.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.204+0000 7fa45cff9640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa44c06b500 0x7fa44c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:34.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.204+0000 7fa45ffff640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa44c06b500 0x7fa44c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:34.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.204+0000 7fa45ffff640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa44c06b500 0x7fa44c06d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4500011e0 tx=0x7fa450007d30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:34.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.207+0000 7fa45cff9640 1 -- 172.21.15.133:0/3386231033 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa458079fb0 con 0x7fa4600c9830 2024-02-21T23:25:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.446+0000 7fa45cff9640 1 -- 172.21.15.133:0/3386231033 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fa45807bf10 con 0x7fa4600c9830 2024-02-21T23:25:34.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.636+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa46014ff70 con 0x7fa4600c9830 2024-02-21T23:25:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.641+0000 7fa45cff9640 1 -- 172.21.15.133:0/3386231033 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa45807ffb0 con 0x7fa4600c9830 2024-02-21T23:25:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-02-21T23:25:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa44c06b500 msgr2=0x7fa44c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa44c06b500 0x7fa44c06d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4500011e0 tx=0x7fa450007d30 comp rx=0 tx=0).stop 2024-02-21T23:25:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600c9830 msgr2=0x7fa4601588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600c9830 0x7fa4601588e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa46006cfb0 tx=0x7fa458004a40 comp rx=0 tx=0).stop 2024-02-21T23:25:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 shutdown_connections 2024-02-21T23:25:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa44c06b500 0x7fa44c06d9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600c9830 0x7fa4601588e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4600c9450 0x7fa4600cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 --2- 172.21.15.133:0/3386231033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4600c7790 0x7fa46015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 >> 172.21.15.133:0/3386231033 conn(0x7fa46006bfd0 msgr2=0x7fa4600cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 shutdown_connections 2024-02-21T23:25:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.643+0000 7fa466a1c640 1 -- 172.21.15.133:0/3386231033 wait complete. 2024-02-21T23:25:34.640 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:25:34.641 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-21T23:25:34.656 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:34.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.820+0000 7f61fe775640 1 Processor -- start 2024-02-21T23:25:34.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.820+0000 7f61fe775640 1 -- start start 2024-02-21T23:25:34.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.821+0000 7f61fe775640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61f806b510 0x7f61f8072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:34.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.821+0000 7f61fe775640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f815a190 con 0x7f61f806b130 2024-02-21T23:25:34.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.821+0000 7f61fe775640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f815a430 con 0x7f61f8154670 2024-02-21T23:25:34.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.821+0000 7f61fe775640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f815a6f0 con 0x7f61f806b510 2024-02-21T23:25:34.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.821+0000 7f61fcd12640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61f806b510 0x7f61f8072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:34.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.821+0000 7f61f77fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f61f8154670 0x7f61f815d9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:54848/0 (socket says 172.21.15.133:54848) 2024-02-21T23:25:34.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.821+0000 7f61f77fe640 1 -- 172.21.15.133:0/615871961 learned_addr learned my addr 172.21.15.133:0/615871961 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:34.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.824+0000 7f61f6ffd640 1 -- 172.21.15.133:0/615871961 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96463223 0 0) 0x7f61f815a430 con 0x7f61f8154670 2024-02-21T23:25:34.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.824+0000 7f61f6ffd640 1 -- 172.21.15.133:0/615871961 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e0003520 con 0x7f61f8154670 2024-02-21T23:25:34.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.826+0000 7f61fcd12640 1 -- 172.21.15.133:0/615871961 >> v1:172.21.15.142:6789/0 conn(0x7f61f8154670 legacy=0x7f61f815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.826+0000 7f61fcd12640 1 -- 172.21.15.133:0/615871961 >> v1:172.21.15.133:6789/0 conn(0x7f61f806b130 legacy=0x7f61f8153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.826+0000 7f61fcd12640 1 -- 172.21.15.133:0/615871961 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f815a9d0 con 0x7f61f806b510 2024-02-21T23:25:34.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.831+0000 7f61fcd12640 1 --2- 172.21.15.133:0/615871961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b510 0x7f61f8072a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009d90 tx=0x7f61f002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ac9724e13d6f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:34.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.835+0000 7f61f6ffd640 1 -- 172.21.15.133:0/615871961 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0039070 con 0x7f61f806b510 2024-02-21T23:25:34.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.835+0000 7f61f6ffd640 1 -- 172.21.15.133:0/615871961 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f002cd80 con 0x7f61f806b510 2024-02-21T23:25:34.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.835+0000 7f61f6ffd640 1 -- 172.21.15.133:0/615871961 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0034660 con 0x7f61f806b510 2024-02-21T23:25:34.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.836+0000 7f61fe775640 1 -- 172.21.15.133:0/615871961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b510 msgr2=0x7f61f8072a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.836+0000 7f61fe775640 1 --2- 172.21.15.133:0/615871961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b510 0x7f61f8072a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009d90 tx=0x7f61f002bfd0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.836+0000 7f61fe775640 1 -- 172.21.15.133:0/615871961 shutdown_connections 2024-02-21T23:25:34.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.836+0000 7f61fe775640 1 --2- 172.21.15.133:0/615871961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b510 0x7f61f8072a60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.836+0000 7f61fe775640 1 -- 172.21.15.133:0/615871961 >> 172.21.15.133:0/615871961 conn(0x7f61f806bfd0 msgr2=0x7f61f806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:34.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.836+0000 7f61fe775640 1 -- 172.21.15.133:0/615871961 shutdown_connections 2024-02-21T23:25:34.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.836+0000 7f61fe775640 1 -- 172.21.15.133:0/615871961 wait complete. 2024-02-21T23:25:34.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.837+0000 7f61fe775640 1 Processor -- start 2024-02-21T23:25:34.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.838+0000 7f61fe775640 1 -- start start 2024-02-21T23:25:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.838+0000 7f61fe775640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b130 0x7f61f81e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.838+0000 7f61fe775640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f806b510 0x7f61f80c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.838+0000 7f61fe775640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61f8154670 0x7f61f80c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.838+0000 7f61fe775640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80704d0 con 0x7f61f806b130 2024-02-21T23:25:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.838+0000 7f61fe775640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f815af90 con 0x7f61f806b510 2024-02-21T23:25:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.838+0000 7f61fe775640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80c78b0 con 0x7f61f8154670 2024-02-21T23:25:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.839+0000 7f61f7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b130 0x7f61f81e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.839+0000 7f61fcd12640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61f8154670 0x7f61f80c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.839+0000 7f61f77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f806b510 0x7f61f80c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.839+0000 7f61f7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b130 0x7f61f81e5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50656/0 (socket says 172.21.15.133:50656) 2024-02-21T23:25:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.839+0000 7f61f7fff640 1 -- 172.21.15.133:0/1598297626 learned_addr learned my addr 172.21.15.133:0/1598297626 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:34.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.842+0000 7f61f77fe640 1 -- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61f8154670 msgr2=0x7f61f80c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.842+0000 7f61f77fe640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61f8154670 0x7f61f80c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.842+0000 7f61f77fe640 1 -- 172.21.15.133:0/1598297626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b130 msgr2=0x7f61f81e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:34.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.842+0000 7f61f77fe640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b130 0x7f61f81e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:34.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.843+0000 7f61f77fe640 1 -- 172.21.15.133:0/1598297626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f81e6570 con 0x7f61f806b510 2024-02-21T23:25:34.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.845+0000 7f61f77fe640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f806b510 0x7f61f80c53d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00c420 tx=0x7f61ec001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:34.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.849+0000 7f61f4ff9640 1 -- 172.21.15.133:0/1598297626 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec017070 con 0x7f61f806b510 2024-02-21T23:25:34.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.849+0000 7f61f4ff9640 1 -- 172.21.15.133:0/1598297626 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec005ce0 con 0x7f61f806b510 2024-02-21T23:25:34.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.849+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f81523f0 con 0x7f61f806b510 2024-02-21T23:25:34.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.849+0000 7f61f4ff9640 1 -- 172.21.15.133:0/1598297626 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec0072d0 con 0x7f61f806b510 2024-02-21T23:25:34.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.849+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f81586c0 con 0x7f61f806b510 2024-02-21T23:25:34.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:34.850+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f8151d60 con 0x7f61f806b510 2024-02-21T23:25:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.102+0000 7f61f4ff9640 1 -- 172.21.15.133:0/1598297626 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61ec01a070 con 0x7f61f806b510 2024-02-21T23:25:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.102+0000 7f61f4ff9640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f61e406b4d0 0x7f61e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.103+0000 7f61f7fff640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f61e406b4d0 0x7f61e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:35.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.103+0000 7f61f7fff640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f61e406b4d0 0x7f61e406d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61e8001840 tx=0x7f61e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:35.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.106+0000 7f61f4ff9640 1 -- 172.21.15.133:0/1598297626 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f61ec0562a0 con 0x7f61f806b510 2024-02-21T23:25:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.117+0000 7f61f4ff9640 1 -- 172.21.15.133:0/1598297626 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f61ec058200 con 0x7f61f806b510 2024-02-21T23:25:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.290+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61f806a130 con 0x7f61f806b510 2024-02-21T23:25:35.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.330+0000 7f61f4ff9640 1 -- 172.21.15.133:0/1598297626 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f61ec05c2a0 con 0x7f61f806b510 2024-02-21T23:25:35.317 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:35.317 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:35.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f61e406b4d0 msgr2=0x7f61e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:35.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f61e406b4d0 0x7f61e406d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61e8001840 tx=0x7f61e8009040 comp rx=0 tx=0).stop 2024-02-21T23:25:35.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f806b510 msgr2=0x7f61f80c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:35.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f806b510 0x7f61f80c53d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00c420 tx=0x7f61ec001e70 comp rx=0 tx=0).stop 2024-02-21T23:25:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 shutdown_connections 2024-02-21T23:25:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f61e406b4d0 0x7f61e406d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61f8154670 0x7f61f80c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f806b510 0x7f61f80c53d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 --2- 172.21.15.133:0/1598297626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f806b130 0x7f61f81e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 >> 172.21.15.133:0/1598297626 conn(0x7f61f806bfd0 msgr2=0x7f61f81591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 shutdown_connections 2024-02-21T23:25:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.332+0000 7f61fe775640 1 -- 172.21.15.133:0/1598297626 wait complete. 2024-02-21T23:25:35.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-21T23:25:02.902698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-21T23:25:13.993803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-21T23:25:15.604077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-21T23:25:19.468744+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-21T23:25:35.330 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-21T23:25:35.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.447+0000 7fe51e6b0640 1 Processor -- start 2024-02-21T23:25:35.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.447+0000 7fe51e6b0640 1 -- start start 2024-02-21T23:25:35.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.448+0000 7fe51e6b0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe518154450 0x7fe51806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:35.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.448+0000 7fe51e6b0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe51815a290 0x7fe5180c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:35.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.448+0000 7fe51e6b0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51815a720 con 0x7fe51806ff00 2024-02-21T23:25:35.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.448+0000 7fe51e6b0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51806c2f0 con 0x7fe51815a290 2024-02-21T23:25:35.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.448+0000 7fe51e6b0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51815a9e0 con 0x7fe518154450 2024-02-21T23:25:35.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.449+0000 7fe517fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe518154450 0x7fe51806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:35.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.449+0000 7fe5177fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe51815a290 0x7fe5180c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:35.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.449+0000 7fe517fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe518154450 0x7fe51806f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:46882/0 (socket says 172.21.15.133:46882) 2024-02-21T23:25:35.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.449+0000 7fe517fff640 1 -- 172.21.15.133:0/4291753282 learned_addr learned my addr 172.21.15.133:0/4291753282 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:35.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.451+0000 7fe517fff640 1 -- 172.21.15.133:0/4291753282 >> v2:172.21.15.133:3300/0 conn(0x7fe51815a290 msgr2=0x7fe5180c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:35.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.451+0000 7fe517fff640 1 --2- 172.21.15.133:0/4291753282 >> v2:172.21.15.133:3300/0 conn(0x7fe51815a290 0x7fe5180c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.451+0000 7fe517fff640 1 -- 172.21.15.133:0/4291753282 >> v1:172.21.15.133:6789/0 conn(0x7fe51806ff00 legacy=0x7fe5180c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:35.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.451+0000 7fe517fff640 1 -- 172.21.15.133:0/4291753282 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51815acc0 con 0x7fe518154450 2024-02-21T23:25:35.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.454+0000 7fe517fff640 1 --2- 172.21.15.133:0/4291753282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518154450 0x7fe51806f9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe51806e1b0 tx=0x7fe50c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcaed7028294438c server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:35.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.457+0000 7fe516ffd640 1 -- 172.21.15.133:0/4291753282 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c004230 con 0x7fe518154450 2024-02-21T23:25:35.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.457+0000 7fe516ffd640 1 -- 172.21.15.133:0/4291753282 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c0043d0 con 0x7fe518154450 2024-02-21T23:25:35.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.457+0000 7fe516ffd640 1 -- 172.21.15.133:0/4291753282 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c030840 con 0x7fe518154450 2024-02-21T23:25:35.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.458+0000 7fe51e6b0640 1 -- 172.21.15.133:0/4291753282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518154450 msgr2=0x7fe51806f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:35.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.458+0000 7fe51e6b0640 1 --2- 172.21.15.133:0/4291753282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518154450 0x7fe51806f9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe51806e1b0 tx=0x7fe50c02b980 comp rx=0 tx=0).stop 2024-02-21T23:25:35.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.458+0000 7fe51e6b0640 1 -- 172.21.15.133:0/4291753282 shutdown_connections 2024-02-21T23:25:35.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.458+0000 7fe51e6b0640 1 --2- 172.21.15.133:0/4291753282 >> v2:172.21.15.133:3300/0 conn(0x7fe51815a290 0x7fe5180c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.458+0000 7fe51e6b0640 1 --2- 172.21.15.133:0/4291753282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518154450 0x7fe51806f9c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.458+0000 7fe51e6b0640 1 -- 172.21.15.133:0/4291753282 >> 172.21.15.133:0/4291753282 conn(0x7fe518070ec0 msgr2=0x7fe5180712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:35.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.458+0000 7fe51e6b0640 1 -- 172.21.15.133:0/4291753282 shutdown_connections 2024-02-21T23:25:35.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.458+0000 7fe51e6b0640 1 -- 172.21.15.133:0/4291753282 wait complete. 2024-02-21T23:25:35.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.459+0000 7fe51e6b0640 1 Processor -- start 2024-02-21T23:25:35.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.460+0000 7fe51e6b0640 1 -- start start 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe51e6b0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51806ff00 0x7fe5181efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe51e6b0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe518154450 0x7fe51814f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe51e6b0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518148ca0 0x7fe518146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe51e6b0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180c3d50 con 0x7fe518154450 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe51e6b0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51806f4a0 con 0x7fe518148ca0 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe51e6b0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180c3ed0 con 0x7fe51806ff00 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe517fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51806ff00 0x7fe5181efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe51cc4d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518148ca0 0x7fe518146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe517fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51806ff00 0x7fe5181efcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:34010/0 (socket says 172.21.15.133:34010) 2024-02-21T23:25:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.461+0000 7fe517fff640 1 -- 172.21.15.133:0/2443759172 learned_addr learned my addr 172.21.15.133:0/2443759172 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:35.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.463+0000 7fe5177fe640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe518154450 0x7fe51814f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:35.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.465+0000 7fe51cc4d640 1 -- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51806ff00 msgr2=0x7fe5181efcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:35.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.465+0000 7fe51cc4d640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51806ff00 0x7fe5181efcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.465+0000 7fe51cc4d640 1 -- 172.21.15.133:0/2443759172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe518154450 msgr2=0x7fe51814f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:35.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.465+0000 7fe51cc4d640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe518154450 0x7fe51814f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.465+0000 7fe51cc4d640 1 -- 172.21.15.133:0/2443759172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5180ca9b0 con 0x7fe518148ca0 2024-02-21T23:25:35.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.469+0000 7fe51cc4d640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518148ca0 0x7fe518146de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c7b0 tx=0x7fe51000cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:35.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.473+0000 7fe514ff9640 1 -- 172.21.15.133:0/2443759172 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510016020 con 0x7fe518148ca0 2024-02-21T23:25:35.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.473+0000 7fe514ff9640 1 -- 172.21.15.133:0/2443759172 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51000aa50 con 0x7fe518148ca0 2024-02-21T23:25:35.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.473+0000 7fe514ff9640 1 -- 172.21.15.133:0/2443759172 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000ce60 con 0x7fe518148ca0 2024-02-21T23:25:35.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.473+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5180cac90 con 0x7fe518148ca0 2024-02-21T23:25:35.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.473+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5181f36c0 con 0x7fe518148ca0 2024-02-21T23:25:35.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.474+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51806a130 con 0x7fe518148ca0 2024-02-21T23:25:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.494+0000 7fe514ff9640 1 -- 172.21.15.133:0/2443759172 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe510019020 con 0x7fe518148ca0 2024-02-21T23:25:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.495+0000 7fe514ff9640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe50406b550 0x7fe50406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:35.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.495+0000 7fe517fff640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe50406b550 0x7fe50406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:35.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.496+0000 7fe517fff640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe50406b550 0x7fe50406da10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0090b0 tx=0x7fe50c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:35.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.497+0000 7fe514ff9640 1 -- 172.21.15.133:0/2443759172 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe510056c80 con 0x7fe518148ca0 2024-02-21T23:25:35.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.746+0000 7fe514ff9640 1 -- 172.21.15.133:0/2443759172 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fe510020030 con 0x7fe518148ca0 2024-02-21T23:25:35.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.939+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe518149f10 con 0x7fe518148ca0 2024-02-21T23:25:35.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.945+0000 7fe514ff9640 1 -- 172.21.15.133:0/2443759172 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe510058be0 con 0x7fe518148ca0 2024-02-21T23:25:35.931 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-02-21T23:25:35.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe50406b550 msgr2=0x7fe50406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:35.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe50406b550 0x7fe50406da10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0090b0 tx=0x7fe50c008030 comp rx=0 tx=0).stop 2024-02-21T23:25:35.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518148ca0 msgr2=0x7fe518146de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:35.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518148ca0 0x7fe518146de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c7b0 tx=0x7fe51000cc70 comp rx=0 tx=0).stop 2024-02-21T23:25:35.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 shutdown_connections 2024-02-21T23:25:35.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe50406b550 0x7fe50406da10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe518148ca0 0x7fe518146de0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe518154450 0x7fe51814f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 --2- 172.21.15.133:0/2443759172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe51806ff00 0x7fe5181efcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:35.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 >> 172.21.15.133:0/2443759172 conn(0x7fe518070ec0 msgr2=0x7fe518159900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:35.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 shutdown_connections 2024-02-21T23:25:35.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:35.947+0000 7fe51e6b0640 1 -- 172.21.15.133:0/2443759172 wait complete. 2024-02-21T23:25:35.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-21T23:25:36.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.061+0000 7ff1c9a6d640 1 Processor -- start 2024-02-21T23:25:36.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.062+0000 7ff1c9a6d640 1 -- start start 2024-02-21T23:25:36.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.062+0000 7ff1c9a6d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff1c4154600 0x7ff1c40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.062+0000 7ff1c9a6d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c7c10 0x7ff1c40c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.062+0000 7ff1c9a6d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c40c85e0 con 0x7ff1c40c6f50 2024-02-21T23:25:36.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.062+0000 7ff1c9a6d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c40c74a0 con 0x7ff1c4154600 2024-02-21T23:25:36.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.062+0000 7ff1c9a6d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40c7780 con 0x7ff1c40c7c10 2024-02-21T23:25:36.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.063+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c7c10 0x7ff1c40c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.063+0000 7ff1c2ffd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7ff1c40c6f50 0x7ff1c40ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:54854/0 (socket says 172.21.15.133:54854) 2024-02-21T23:25:36.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.063+0000 7ff1c2ffd640 1 -- 172.21.15.133:0/3705840949 learned_addr learned my addr 172.21.15.133:0/3705840949 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:36.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.064+0000 7ff1c37fe640 1 --2- 172.21.15.133:0/3705840949 >> v2:172.21.15.142:3300/0 conn(0x7ff1c4154600 0x7ff1c40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.066+0000 7ff1c3fff640 1 -- 172.21.15.133:0/3705840949 >> v2:172.21.15.142:3300/0 conn(0x7ff1c4154600 msgr2=0x7ff1c40cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.066+0000 7ff1c3fff640 1 --2- 172.21.15.133:0/3705840949 >> v2:172.21.15.142:3300/0 conn(0x7ff1c4154600 0x7ff1c40cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.066+0000 7ff1c3fff640 1 -- 172.21.15.133:0/3705840949 >> v1:172.21.15.142:6789/0 conn(0x7ff1c40c6f50 legacy=0x7ff1c40ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.066+0000 7ff1c3fff640 1 -- 172.21.15.133:0/3705840949 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c415ab40 con 0x7ff1c40c7c10 2024-02-21T23:25:36.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.068+0000 7ff1c3fff640 1 --2- 172.21.15.133:0/3705840949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c7c10 0x7ff1c40c80a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009d90 tx=0x7ff1bc02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab305d813bd6c713 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.071+0000 7ff1c27fc640 1 -- 172.21.15.133:0/3705840949 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc039070 con 0x7ff1c40c7c10 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.071+0000 7ff1c27fc640 1 -- 172.21.15.133:0/3705840949 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc02cc50 con 0x7ff1c40c7c10 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.071+0000 7ff1c27fc640 1 -- 172.21.15.133:0/3705840949 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc034440 con 0x7ff1c40c7c10 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.072+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/3705840949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c7c10 msgr2=0x7ff1c40c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.072+0000 7ff1c9a6d640 1 --2- 172.21.15.133:0/3705840949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c7c10 0x7ff1c40c80a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009d90 tx=0x7ff1bc02bea0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.072+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/3705840949 shutdown_connections 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.072+0000 7ff1c9a6d640 1 --2- 172.21.15.133:0/3705840949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c7c10 0x7ff1c40c80a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.072+0000 7ff1c9a6d640 1 --2- 172.21.15.133:0/3705840949 >> v2:172.21.15.142:3300/0 conn(0x7ff1c4154600 0x7ff1c40cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.072+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/3705840949 >> 172.21.15.133:0/3705840949 conn(0x7ff1c4062000 msgr2=0x7ff1c40cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.072+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/3705840949 shutdown_connections 2024-02-21T23:25:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.072+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/3705840949 wait complete. 2024-02-21T23:25:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.073+0000 7ff1c9a6d640 1 Processor -- start 2024-02-21T23:25:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.073+0000 7ff1c9a6d640 1 -- start start 2024-02-21T23:25:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.074+0000 7ff1c9a6d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c6f50 0x7ff1c41429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.074+0000 7ff1c9a6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c40c7c10 0x7ff1c4149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.074+0000 7ff1c9a6d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1c4154600 0x7ff1c41446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.074+0000 7ff1c9a6d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40cb860 con 0x7ff1c40c7c10 2024-02-21T23:25:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.074+0000 7ff1c9a6d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40cb580 con 0x7ff1c4154600 2024-02-21T23:25:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.074+0000 7ff1c9a6d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40b5720 con 0x7ff1c40c6f50 2024-02-21T23:25:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.075+0000 7ff1c37fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c6f50 0x7ff1c41429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.075+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1c4154600 0x7ff1c41446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.075+0000 7ff1c2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c40c7c10 0x7ff1c4149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.075+0000 7ff1c37fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c6f50 0x7ff1c41429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:34036/0 (socket says 172.21.15.133:34036) 2024-02-21T23:25:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.075+0000 7ff1c37fe640 1 -- 172.21.15.133:0/2540489367 learned_addr learned my addr 172.21.15.133:0/2540489367 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.078+0000 7ff1c2ffd640 1 -- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c6f50 msgr2=0x7ff1c41429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.078+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c6f50 0x7ff1c41429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.078+0000 7ff1c2ffd640 1 -- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1c4154600 msgr2=0x7ff1c41446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.078+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1c4154600 0x7ff1c41446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.079+0000 7ff1c2ffd640 1 -- 172.21.15.133:0/2540489367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c414a9d0 con 0x7ff1c40c7c10 2024-02-21T23:25:36.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.081+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c40c7c10 0x7ff1c4149250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c370 tx=0x7ff1b800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.084+0000 7ff1abfff640 1 -- 172.21.15.133:0/2540489367 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8017070 con 0x7ff1c40c7c10 2024-02-21T23:25:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.085+0000 7ff1abfff640 1 -- 172.21.15.133:0/2540489367 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b8005ce0 con 0x7ff1c40c7c10 2024-02-21T23:25:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.085+0000 7ff1abfff640 1 -- 172.21.15.133:0/2540489367 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8007310 con 0x7ff1c40c7c10 2024-02-21T23:25:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.085+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c4143380 con 0x7ff1c40c7c10 2024-02-21T23:25:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.085+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c4143900 con 0x7ff1c40c7c10 2024-02-21T23:25:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.086+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c4068300 con 0x7ff1c40c7c10 2024-02-21T23:25:36.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.103+0000 7ff1abfff640 1 -- 172.21.15.133:0/2540489367 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1b801a070 con 0x7ff1c40c7c10 2024-02-21T23:25:36.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.104+0000 7ff1abfff640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff1b006b520 0x7ff1b006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.104+0000 7ff1c37fe640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff1b006b520 0x7ff1b006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.104+0000 7ff1c37fe640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff1b006b520 0x7ff1b006d9e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4007a60 tx=0x7ff1b4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:36.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.107+0000 7ff1abfff640 1 -- 172.21.15.133:0/2540489367 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff1b80561b0 con 0x7ff1c40c7c10 2024-02-21T23:25:36.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.342+0000 7ff1abfff640 1 -- 172.21.15.133:0/2540489367 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7ff1c4068300 con 0x7ff1c40c7c10 2024-02-21T23:25:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.513+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1c406a130 con 0x7ff1c40c7c10 2024-02-21T23:25:36.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.519+0000 7ff1abfff640 1 -- 172.21.15.133:0/2540489367 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1b8021020 con 0x7ff1c40c7c10 2024-02-21T23:25:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:25:36.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff1b006b520 msgr2=0x7ff1b006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:36.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff1b006b520 0x7ff1b006d9e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4007a60 tx=0x7ff1b4008040 comp rx=0 tx=0).stop 2024-02-21T23:25:36.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c40c7c10 msgr2=0x7ff1c4149250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:36.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c40c7c10 0x7ff1c4149250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c370 tx=0x7ff1b800c830 comp rx=0 tx=0).stop 2024-02-21T23:25:36.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 shutdown_connections 2024-02-21T23:25:36.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7ff1b006b520 0x7ff1b006d9e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1c4154600 0x7ff1c41446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c40c7c10 0x7ff1c4149250 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 --2- 172.21.15.133:0/2540489367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1c40c6f50 0x7ff1c41429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 >> 172.21.15.133:0/2540489367 conn(0x7ff1c4062000 msgr2=0x7ff1c4149f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:36.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 shutdown_connections 2024-02-21T23:25:36.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.521+0000 7ff1c9a6d640 1 -- 172.21.15.133:0/2540489367 wait complete. 2024-02-21T23:25:36.519 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-21T23:25:36.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.635+0000 7f65a6036640 1 Processor -- start 2024-02-21T23:25:36.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.635+0000 7f65a6036640 1 -- start start 2024-02-21T23:25:36.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.636+0000 7f65a6036640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65a0154120 0x7f65a00c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.636+0000 7f65a6036640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a00c4600 0x7f65a015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.636+0000 7f65a6036640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f65a015ac10 0x7f65a00cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.636+0000 7f65a6036640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a00c3760 con 0x7f65a0154120 2024-02-21T23:25:36.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.636+0000 7f65a6036640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a00c3a00 con 0x7f65a015ac10 2024-02-21T23:25:36.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.636+0000 7f65a6036640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00c3ce0 con 0x7f65a00c4600 2024-02-21T23:25:36.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.636+0000 7f659f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65a0154120 0x7f65a00c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.637+0000 7f659ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f65a015ac10 0x7f65a00cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.637+0000 7f6597fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a00c4600 0x7f65a015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.637+0000 7f659f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65a0154120 0x7f65a00c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50696/0 (socket says 172.21.15.133:50696) 2024-02-21T23:25:36.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.637+0000 7f659f7fe640 1 -- 172.21.15.133:0/3976257853 learned_addr learned my addr 172.21.15.133:0/3976257853 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:36.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.639+0000 7f659f7fe640 1 -- 172.21.15.133:0/3976257853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a00c4600 msgr2=0x7f65a015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:36.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.639+0000 7f659f7fe640 1 --2- 172.21.15.133:0/3976257853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a00c4600 0x7f65a015a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.639+0000 7f659f7fe640 1 -- 172.21.15.133:0/3976257853 >> v2:172.21.15.142:3300/0 conn(0x7f65a015ac10 msgr2=0x7f65a00cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:36.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.639+0000 7f659f7fe640 1 --2- 172.21.15.133:0/3976257853 >> v2:172.21.15.142:3300/0 conn(0x7f65a015ac10 0x7f65a00cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.639+0000 7f659f7fe640 1 -- 172.21.15.133:0/3976257853 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00c3fc0 con 0x7f65a0154120 2024-02-21T23:25:36.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.641+0000 7f659f7fe640 1 --2- 172.21.15.133:0/3976257853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65a0154120 0x7f65a00c8900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f65a015e630 tx=0x7f659002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7057461d224e551 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:36.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f659effd640 1 -- 172.21.15.133:0/3976257853 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590034070 con 0x7f65a0154120 2024-02-21T23:25:36.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f659effd640 1 -- 172.21.15.133:0/3976257853 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590030860 con 0x7f65a0154120 2024-02-21T23:25:36.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f659effd640 1 -- 172.21.15.133:0/3976257853 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590030c30 con 0x7f65a0154120 2024-02-21T23:25:36.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f65a6036640 1 -- 172.21.15.133:0/3976257853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65a0154120 msgr2=0x7f65a00c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:36.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f65a6036640 1 --2- 172.21.15.133:0/3976257853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65a0154120 0x7f65a00c8900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f65a015e630 tx=0x7f659002b9a0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f65a6036640 1 -- 172.21.15.133:0/3976257853 shutdown_connections 2024-02-21T23:25:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f65a6036640 1 --2- 172.21.15.133:0/3976257853 >> v2:172.21.15.142:3300/0 conn(0x7f65a015ac10 0x7f65a00cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f65a6036640 1 --2- 172.21.15.133:0/3976257853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a00c4600 0x7f65a015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f65a6036640 1 --2- 172.21.15.133:0/3976257853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65a0154120 0x7f65a00c8900 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f65a6036640 1 -- 172.21.15.133:0/3976257853 >> 172.21.15.133:0/3976257853 conn(0x7f65a006f7f0 msgr2=0x7f65a0061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f65a6036640 1 -- 172.21.15.133:0/3976257853 shutdown_connections 2024-02-21T23:25:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.645+0000 7f65a6036640 1 -- 172.21.15.133:0/3976257853 wait complete. 2024-02-21T23:25:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.647+0000 7f65a6036640 1 Processor -- start 2024-02-21T23:25:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.647+0000 7f65a6036640 1 -- start start 2024-02-21T23:25:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.647+0000 7f65a6036640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65a00c4600 0x7f65a00c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.647+0000 7f65a6036640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a0154120 0x7f65a014f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.647+0000 7f65a6036640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65a015ac10 0x7f65a01e3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.647+0000 7f65a6036640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00c4ba0 con 0x7f65a015ac10 2024-02-21T23:25:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.647+0000 7f65a6036640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00c4a20 con 0x7f65a00c4600 2024-02-21T23:25:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.647+0000 7f65a6036640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0154500 con 0x7f65a0154120 2024-02-21T23:25:36.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.648+0000 7f659ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65a015ac10 0x7f65a01e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.648+0000 7f659f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65a00c4600 0x7f65a00c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.648+0000 7f6597fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a0154120 0x7f65a014f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.648+0000 7f659f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65a00c4600 0x7f65a00c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:46932/0 (socket says 172.21.15.133:46932) 2024-02-21T23:25:36.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.648+0000 7f659f7fe640 1 -- 172.21.15.133:0/3892348989 learned_addr learned my addr 172.21.15.133:0/3892348989 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:36.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.651+0000 7f659f7fe640 1 -- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a0154120 msgr2=0x7f65a014f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:36.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.651+0000 7f659f7fe640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a0154120 0x7f65a014f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.651+0000 7f659f7fe640 1 -- 172.21.15.133:0/3892348989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65a015ac10 msgr2=0x7f65a01e3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:36.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.651+0000 7f659f7fe640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65a015ac10 0x7f65a01e3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:36.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.651+0000 7f659f7fe640 1 -- 172.21.15.133:0/3892348989 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0150160 con 0x7f65a00c4600 2024-02-21T23:25:36.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.654+0000 7f659f7fe640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65a00c4600 0x7f65a00c5dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6590000c00 tx=0x7f6590004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:36.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.657+0000 7f659cff9640 1 -- 172.21.15.133:0/3892348989 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590048070 con 0x7f65a00c4600 2024-02-21T23:25:36.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.657+0000 7f659cff9640 1 -- 172.21.15.133:0/3892348989 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590030660 con 0x7f65a00c4600 2024-02-21T23:25:36.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.657+0000 7f659cff9640 1 -- 172.21.15.133:0/3892348989 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590034040 con 0x7f65a00c4600 2024-02-21T23:25:36.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.657+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a0150440 con 0x7f65a00c4600 2024-02-21T23:25:36.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.658+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a014e840 con 0x7f65a00c4600 2024-02-21T23:25:36.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.658+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a006a130 con 0x7f65a00c4600 2024-02-21T23:25:36.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.942+0000 7f659cff9640 1 -- 172.21.15.133:0/3892348989 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f659004b070 con 0x7f65a00c4600 2024-02-21T23:25:36.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.942+0000 7f659cff9640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f658006b550 0x7f658006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:36.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.942+0000 7f6597fff640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f658006b550 0x7f658006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:36.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.942+0000 7f6597fff640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f658006b550 0x7f658006da10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65a0153b30 tx=0x7f6588009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:36.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.945+0000 7f659cff9640 1 -- 172.21.15.133:0/3892348989 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6590044e00 con 0x7f65a00c4600 2024-02-21T23:25:36.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:36.958+0000 7f659cff9640 1 -- 172.21.15.133:0/3892348989 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f6590052020 con 0x7f65a00c4600 2024-02-21T23:25:37.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.135+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65a00cb0e0 con 0x7f65a00c4600 2024-02-21T23:25:37.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.158+0000 7f659cff9640 1 -- 172.21.15.133:0/3892348989 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6590030830 con 0x7f65a00c4600 2024-02-21T23:25:37.145 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-02-21T23:25:37.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f658006b550 msgr2=0x7f658006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f658006b550 0x7f658006da10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65a0153b30 tx=0x7f6588009190 comp rx=0 tx=0).stop 2024-02-21T23:25:37.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65a00c4600 msgr2=0x7f65a00c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65a00c4600 0x7f65a00c5dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6590000c00 tx=0x7f6590004730 comp rx=0 tx=0).stop 2024-02-21T23:25:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 shutdown_connections 2024-02-21T23:25:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f658006b550 0x7f658006da10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65a015ac10 0x7f65a01e3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65a0154120 0x7f65a014f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 --2- 172.21.15.133:0/3892348989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65a00c4600 0x7f65a00c5dc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 >> 172.21.15.133:0/3892348989 conn(0x7f65a006f7f0 msgr2=0x7f65a006d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 shutdown_connections 2024-02-21T23:25:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.160+0000 7f65a6036640 1 -- 172.21.15.133:0/3892348989 wait complete. 2024-02-21T23:25:37.158 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-21T23:25:37.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.274+0000 7fb2cb11e640 1 Processor -- start 2024-02-21T23:25:37.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.274+0000 7fb2cb11e640 1 -- start start 2024-02-21T23:25:37.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.275+0000 7fb2cb11e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2c4154450 0x7fb2c406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.275+0000 7fb2cb11e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2c415a290 0x7fb2c40c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.275+0000 7fb2cb11e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c406ff00 0x7fb2c40c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.275+0000 7fb2cb11e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c406c2f0 con 0x7fb2c415a290 2024-02-21T23:25:37.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.275+0000 7fb2cb11e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c415a720 con 0x7fb2c4154450 2024-02-21T23:25:37.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.275+0000 7fb2cb11e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415a9e0 con 0x7fb2c406ff00 2024-02-21T23:25:37.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.275+0000 7fb2bbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2c415a290 0x7fb2c40c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.275+0000 7fb2c8eba640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2c4154450 0x7fb2c406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.275+0000 7fb2c96bb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c406ff00 0x7fb2c40c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.276+0000 7fb2c8eba640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2c4154450 0x7fb2c406f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:46948/0 (socket says 172.21.15.133:46948) 2024-02-21T23:25:37.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.276+0000 7fb2c96bb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c406ff00 0x7fb2c40c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:34072/0 (socket says 172.21.15.133:34072) 2024-02-21T23:25:37.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.276+0000 7fb2c8eba640 1 -- 172.21.15.133:0/1930942353 learned_addr learned my addr 172.21.15.133:0/1930942353 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:37.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.278+0000 7fb2bbfff640 1 -- 172.21.15.133:0/1930942353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c406ff00 msgr2=0x7fb2c40c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:37.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.278+0000 7fb2bbfff640 1 --2- 172.21.15.133:0/1930942353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c406ff00 0x7fb2c40c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.278+0000 7fb2bbfff640 1 -- 172.21.15.133:0/1930942353 >> v2:172.21.15.142:3300/0 conn(0x7fb2c4154450 msgr2=0x7fb2c406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:37.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.278+0000 7fb2bbfff640 1 --2- 172.21.15.133:0/1930942353 >> v2:172.21.15.142:3300/0 conn(0x7fb2c4154450 0x7fb2c406f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.278+0000 7fb2bbfff640 1 -- 172.21.15.133:0/1930942353 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c415acc0 con 0x7fb2c415a290 2024-02-21T23:25:37.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.280+0000 7fb2bbfff640 1 --2- 172.21.15.133:0/1930942353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c415a290 0x7fb2c40c8230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2c406e1b0 tx=0x7fb2b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aecfa3aac8cf3bd server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:37.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2bb7fe640 1 -- 172.21.15.133:0/1930942353 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4035020 con 0x7fb2c415a290 2024-02-21T23:25:37.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2bb7fe640 1 -- 172.21.15.133:0/1930942353 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b4031830 con 0x7fb2c415a290 2024-02-21T23:25:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2bb7fe640 1 -- 172.21.15.133:0/1930942353 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4031c00 con 0x7fb2c415a290 2024-02-21T23:25:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1930942353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c415a290 msgr2=0x7fb2c40c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1930942353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c415a290 0x7fb2c40c8230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2c406e1b0 tx=0x7fb2b402b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1930942353 shutdown_connections 2024-02-21T23:25:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1930942353 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c406ff00 0x7fb2c40c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1930942353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c415a290 0x7fb2c40c8230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1930942353 >> v2:172.21.15.142:3300/0 conn(0x7fb2c4154450 0x7fb2c406f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1930942353 >> 172.21.15.133:0/1930942353 conn(0x7fb2c4070ec0 msgr2=0x7fb2c40712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1930942353 shutdown_connections 2024-02-21T23:25:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.284+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1930942353 wait complete. 2024-02-21T23:25:37.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.286+0000 7fb2cb11e640 1 Processor -- start 2024-02-21T23:25:37.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.286+0000 7fb2cb11e640 1 -- start start 2024-02-21T23:25:37.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2cb11e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c406ff00 0x7fb2c4144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2cb11e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c4154450 0x7fb2c41f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2cb11e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c40c7bc0 0x7fb2c41f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2cb11e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c4154830 con 0x7fb2c406ff00 2024-02-21T23:25:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2cb11e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c406f4a0 con 0x7fb2c4154450 2024-02-21T23:25:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2cb11e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c41549b0 con 0x7fb2c40c7bc0 2024-02-21T23:25:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2c8eba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c406ff00 0x7fb2c4144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2bbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c4154450 0x7fb2c41f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2c96bb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c40c7bc0 0x7fb2c41f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2bbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c4154450 0x7fb2c41f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:46956/0 (socket says 172.21.15.133:46956) 2024-02-21T23:25:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.287+0000 7fb2bbfff640 1 -- 172.21.15.133:0/1465209396 learned_addr learned my addr 172.21.15.133:0/1465209396 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.291+0000 7fb2c8eba640 1 -- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c40c7bc0 msgr2=0x7fb2c41f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.291+0000 7fb2c8eba640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c40c7bc0 0x7fb2c41f3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.291+0000 7fb2c8eba640 1 -- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c4154450 msgr2=0x7fb2c41f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.291+0000 7fb2c8eba640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c4154450 0x7fb2c41f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.291+0000 7fb2c8eba640 1 -- 172.21.15.133:0/1465209396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c414e6f0 con 0x7fb2c406ff00 2024-02-21T23:25:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.293+0000 7fb2c8eba640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c406ff00 0x7fb2c4144470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000e870 tx=0x7fb2c000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:37.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.296+0000 7fb2b97fa640 1 -- 172.21.15.133:0/1465209396 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0005990 con 0x7fb2c406ff00 2024-02-21T23:25:37.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.296+0000 7fb2b97fa640 1 -- 172.21.15.133:0/1465209396 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c0005b30 con 0x7fb2c406ff00 2024-02-21T23:25:37.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.296+0000 7fb2b97fa640 1 -- 172.21.15.133:0/1465209396 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c000c940 con 0x7fb2c406ff00 2024-02-21T23:25:37.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.296+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c41f41c0 con 0x7fb2c406ff00 2024-02-21T23:25:37.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.296+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c41500e0 con 0x7fb2c406ff00 2024-02-21T23:25:37.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.297+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c406a130 con 0x7fb2c406ff00 2024-02-21T23:25:37.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.316+0000 7fb2b97fa640 1 -- 172.21.15.133:0/1465209396 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2c000cae0 con 0x7fb2c406ff00 2024-02-21T23:25:37.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.316+0000 7fb2b97fa640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb2b006b520 0x7fb2b006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.317+0000 7fb2bbfff640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb2b006b520 0x7fb2b006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.317+0000 7fb2bbfff640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb2b006b520 0x7fb2b006d9e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2c41558a0 tx=0x7fb2b4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:37.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.555+0000 7fb2b97fa640 1 -- 172.21.15.133:0/1465209396 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb2c0057240 con 0x7fb2c406ff00 2024-02-21T23:25:37.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.555+0000 7fb2b97fa640 1 -- 172.21.15.133:0/1465209396 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fb2c0017020 con 0x7fb2c406ff00 2024-02-21T23:25:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.725+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb2c414a0a0 con 0x7fb2c406ff00 2024-02-21T23:25:37.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.730+0000 7fb2b97fa640 1 -- 172.21.15.133:0/1465209396 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb2c00591a0 con 0x7fb2c406ff00 2024-02-21T23:25:37.717 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb2b006b520 msgr2=0x7fb2b006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb2b006b520 0x7fb2b006d9e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2c41558a0 tx=0x7fb2b4008030 comp rx=0 tx=0).stop 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c406ff00 msgr2=0x7fb2c4144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c406ff00 0x7fb2c4144470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000e870 tx=0x7fb2c000ed30 comp rx=0 tx=0).stop 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 shutdown_connections 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb2b006b520 0x7fb2b006d9e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c40c7bc0 0x7fb2c41f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c4154450 0x7fb2c41f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 --2- 172.21.15.133:0/1465209396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c406ff00 0x7fb2c4144470 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 >> 172.21.15.133:0/1465209396 conn(0x7fb2c4070ec0 msgr2=0x7fb2c4158550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 shutdown_connections 2024-02-21T23:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.733+0000 7fb2cb11e640 1 -- 172.21.15.133:0/1465209396 wait complete. 2024-02-21T23:25:37.731 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-21T23:25:37.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.844+0000 7f651d236640 1 Processor -- start 2024-02-21T23:25:37.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.845+0000 7f651d236640 1 -- start start 2024-02-21T23:25:37.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.845+0000 7f651d236640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6518154600 0x7f65180cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.845+0000 7f651d236640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65180c7c10 0x7f65180c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.845+0000 7f651d236640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65180c85e0 con 0x7f65180c6f50 2024-02-21T23:25:37.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.845+0000 7f651d236640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65180c74a0 con 0x7f6518154600 2024-02-21T23:25:37.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.845+0000 7f651d236640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65180c7780 con 0x7f65180c7c10 2024-02-21T23:25:37.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.846+0000 7f6516d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6518154600 0x7f65180cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.846+0000 7f651659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f65180c6f50 0x7f65180ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44856/0 (socket says 172.21.15.133:44856) 2024-02-21T23:25:37.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.846+0000 7f651659c640 1 -- 172.21.15.133:0/1678320549 learned_addr learned my addr 172.21.15.133:0/1678320549 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:37.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.846+0000 7f651759e640 1 --2- 172.21.15.133:0/1678320549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65180c7c10 0x7f65180c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.849+0000 7f6515d9b640 1 -- 172.21.15.133:0/1678320549 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655884065 0 0) 0x7f65180c85e0 con 0x7f65180c6f50 2024-02-21T23:25:37.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.849+0000 7f6515d9b640 1 -- 172.21.15.133:0/1678320549 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6500003520 con 0x7f65180c6f50 2024-02-21T23:25:37.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.849+0000 7f6516d9d640 1 -- 172.21.15.133:0/1678320549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65180c7c10 msgr2=0x7f65180c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:37.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.849+0000 7f6516d9d640 1 --2- 172.21.15.133:0/1678320549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65180c7c10 0x7f65180c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.849+0000 7f6516d9d640 1 -- 172.21.15.133:0/1678320549 >> v1:172.21.15.133:6789/0 conn(0x7f65180c6f50 legacy=0x7f65180ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.849+0000 7f6516d9d640 1 -- 172.21.15.133:0/1678320549 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651815ab40 con 0x7f6518154600 2024-02-21T23:25:37.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.851+0000 7f6516d9d640 1 --2- 172.21.15.133:0/1678320549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6518154600 0x7f65180cbc00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65080097a0 tx=0x7f650802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de59e07f43df97ec server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.854+0000 7f6515d9b640 1 -- 172.21.15.133:0/1678320549 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508035020 con 0x7f6518154600 2024-02-21T23:25:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.854+0000 7f6515d9b640 1 -- 172.21.15.133:0/1678320549 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6508031830 con 0x7f6518154600 2024-02-21T23:25:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.854+0000 7f6515d9b640 1 -- 172.21.15.133:0/1678320549 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508031c00 con 0x7f6518154600 2024-02-21T23:25:37.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.855+0000 7f651d236640 1 -- 172.21.15.133:0/1678320549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6518154600 msgr2=0x7f65180cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.855+0000 7f651d236640 1 --2- 172.21.15.133:0/1678320549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6518154600 0x7f65180cbc00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65080097a0 tx=0x7f650802b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.855+0000 7f651d236640 1 -- 172.21.15.133:0/1678320549 shutdown_connections 2024-02-21T23:25:37.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.855+0000 7f651d236640 1 --2- 172.21.15.133:0/1678320549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65180c7c10 0x7f65180c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.855+0000 7f651d236640 1 --2- 172.21.15.133:0/1678320549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6518154600 0x7f65180cbc00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.855+0000 7f651d236640 1 -- 172.21.15.133:0/1678320549 >> 172.21.15.133:0/1678320549 conn(0x7f6518062000 msgr2=0x7f65180cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:37.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.855+0000 7f651d236640 1 -- 172.21.15.133:0/1678320549 shutdown_connections 2024-02-21T23:25:37.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.855+0000 7f651d236640 1 -- 172.21.15.133:0/1678320549 wait complete. 2024-02-21T23:25:37.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.857+0000 7f651d236640 1 Processor -- start 2024-02-21T23:25:37.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.857+0000 7f651d236640 1 -- start start 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.857+0000 7f651d236640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65180c6f50 0x7f65181f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.857+0000 7f651d236640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65180c7c10 0x7f65181f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.857+0000 7f651d236640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6518154600 0x7f65180c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.857+0000 7f651d236640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65180cb860 con 0x7f65180c7c10 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.857+0000 7f651d236640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65180cb580 con 0x7f65180c6f50 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.857+0000 7f651d236640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65180b5720 con 0x7f6518154600 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.858+0000 7f651759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6518154600 0x7f65180c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.858+0000 7f6516d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65180c6f50 0x7f65181f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.858+0000 7f651659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65180c7c10 0x7f65181f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.858+0000 7f651759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6518154600 0x7f65180c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:34092/0 (socket says 172.21.15.133:34092) 2024-02-21T23:25:37.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.858+0000 7f651759e640 1 -- 172.21.15.133:0/3425699288 learned_addr learned my addr 172.21.15.133:0/3425699288 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:37.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.861+0000 7f651759e640 1 -- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65180c6f50 msgr2=0x7f65181f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.861+0000 7f651759e640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65180c6f50 0x7f65181f00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.861+0000 7f651759e640 1 -- 172.21.15.133:0/3425699288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65180c7c10 msgr2=0x7f65181f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:37.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.861+0000 7f651759e640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65180c7c10 0x7f65181f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:37.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.861+0000 7f651759e640 1 -- 172.21.15.133:0/3425699288 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65180cac30 con 0x7f6518154600 2024-02-21T23:25:37.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.864+0000 7f651759e640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6518154600 0x7f65180c33f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f651000efc0 tx=0x7f651000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:37.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.867+0000 7f64ff7fe640 1 -- 172.21.15.133:0/3425699288 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510018020 con 0x7f6518154600 2024-02-21T23:25:37.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.867+0000 7f64ff7fe640 1 -- 172.21.15.133:0/3425699288 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6510014480 con 0x7f6518154600 2024-02-21T23:25:37.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.867+0000 7f64ff7fe640 1 -- 172.21.15.133:0/3425699288 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510014870 con 0x7f6518154600 2024-02-21T23:25:37.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.867+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65180c3930 con 0x7f6518154600 2024-02-21T23:25:37.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.868+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6518152740 con 0x7f6518154600 2024-02-21T23:25:37.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.868+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65180c9a40 con 0x7f6518154600 2024-02-21T23:25:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.887+0000 7f64ff7fe640 1 -- 172.21.15.133:0/3425699288 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6510005d10 con 0x7f6518154600 2024-02-21T23:25:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.887+0000 7f64ff7fe640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f650406b520 0x7f650406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.887+0000 7f6516d9d640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f650406b520 0x7f650406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.888+0000 7f6516d9d640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f650406b520 0x7f650406d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6508036000 tx=0x7f650803c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:37.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:37.889+0000 7f64ff7fe640 1 -- 172.21.15.133:0/3425699288 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6510027de0 con 0x7f6518154600 2024-02-21T23:25:38.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.134+0000 7f64ff7fe640 1 -- 172.21.15.133:0/3425699288 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f65180c9a40 con 0x7f6518154600 2024-02-21T23:25:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.328+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f651806a130 con 0x7f6518154600 2024-02-21T23:25:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.333+0000 7f64ff7fe640 1 -- 172.21.15.133:0/3425699288 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6510022030 con 0x7f6518154600 2024-02-21T23:25:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f650406b520 msgr2=0x7f650406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f650406b520 0x7f650406d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6508036000 tx=0x7f650803c040 comp rx=0 tx=0).stop 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6518154600 msgr2=0x7f65180c33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6518154600 0x7f65180c33f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f651000efc0 tx=0x7f651000c490 comp rx=0 tx=0).stop 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 shutdown_connections 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f650406b520 0x7f650406d9e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6518154600 0x7f65180c33f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65180c7c10 0x7f65181f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 --2- 172.21.15.133:0/3425699288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65180c6f50 0x7f65181f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 >> 172.21.15.133:0/3425699288 conn(0x7f6518062000 msgr2=0x7f65180ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 shutdown_connections 2024-02-21T23:25:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:38.335+0000 7f651d236640 1 -- 172.21.15.133:0/3425699288 wait complete. 2024-02-21T23:25:38.333 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-02-21T23:25:38.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:38.333 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-02-21T23:25:38.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[42590]: fuse finished with error 0 and tester_r 0 2024-02-21T23:25:38.360 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:25:44.461 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-02-21T23:25:44.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:25:44.462 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-02-21T23:25:44.497 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:44.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.661+0000 7fae454d5640 1 Processor -- start 2024-02-21T23:25:44.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.661+0000 7fae454d5640 1 -- start start 2024-02-21T23:25:44.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.662+0000 7fae454d5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae400c6f50 0x7fae400cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:44.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.662+0000 7fae454d5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4015ab40 con 0x7fae400c7c10 2024-02-21T23:25:44.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.662+0000 7fae454d5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4015ae00 con 0x7fae40154600 2024-02-21T23:25:44.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.662+0000 7fae454d5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae400c74a0 con 0x7fae400c6f50 2024-02-21T23:25:44.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.662+0000 7fae3effd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fae40154600 0x7fae400ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:36068/0 (socket says 172.21.15.133:36068) 2024-02-21T23:25:44.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.662+0000 7fae3effd640 1 -- 172.21.15.133:0/2702951356 learned_addr learned my addr 172.21.15.133:0/2702951356 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:44.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.662+0000 7fae3e7fc640 1 --2- 172.21.15.133:0/2702951356 >> v2:172.21.15.133:3300/0 conn(0x7fae400c6f50 0x7fae400cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.665+0000 7fae3dffb640 1 -- 172.21.15.133:0/2702951356 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902688663 0 0) 0x7fae4015ae00 con 0x7fae40154600 2024-02-21T23:25:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.665+0000 7fae3dffb640 1 -- 172.21.15.133:0/2702951356 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae28003520 con 0x7fae40154600 2024-02-21T23:25:44.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.665+0000 7fae3e7fc640 1 -- 172.21.15.133:0/2702951356 >> v1:172.21.15.142:6789/0 conn(0x7fae40154600 legacy=0x7fae400ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:44.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.665+0000 7fae3e7fc640 1 -- 172.21.15.133:0/2702951356 >> v1:172.21.15.133:6789/0 conn(0x7fae400c7c10 legacy=0x7fae400c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:44.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.665+0000 7fae3e7fc640 1 -- 172.21.15.133:0/2702951356 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400cb580 con 0x7fae400c6f50 2024-02-21T23:25:44.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.668+0000 7fae3e7fc640 1 --2- 172.21.15.133:0/2702951356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c6f50 0x7fae400cbc00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae340097a0 tx=0x7fae3402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0ac850fda1c918 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:44.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.672+0000 7fae3dffb640 1 -- 172.21.15.133:0/2702951356 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34034070 con 0x7fae400c6f50 2024-02-21T23:25:44.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.672+0000 7fae3dffb640 1 -- 172.21.15.133:0/2702951356 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34030610 con 0x7fae400c6f50 2024-02-21T23:25:44.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.672+0000 7fae3dffb640 1 -- 172.21.15.133:0/2702951356 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340309e0 con 0x7fae400c6f50 2024-02-21T23:25:44.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.673+0000 7fae454d5640 1 -- 172.21.15.133:0/2702951356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c6f50 msgr2=0x7fae400cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:44.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.673+0000 7fae454d5640 1 --2- 172.21.15.133:0/2702951356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c6f50 0x7fae400cbc00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae340097a0 tx=0x7fae3402b960 comp rx=0 tx=0).stop 2024-02-21T23:25:44.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.673+0000 7fae454d5640 1 -- 172.21.15.133:0/2702951356 shutdown_connections 2024-02-21T23:25:44.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.673+0000 7fae454d5640 1 --2- 172.21.15.133:0/2702951356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c6f50 0x7fae400cbc00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:44.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.673+0000 7fae454d5640 1 -- 172.21.15.133:0/2702951356 >> 172.21.15.133:0/2702951356 conn(0x7fae40062000 msgr2=0x7fae400cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:44.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.673+0000 7fae454d5640 1 -- 172.21.15.133:0/2702951356 shutdown_connections 2024-02-21T23:25:44.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.673+0000 7fae454d5640 1 -- 172.21.15.133:0/2702951356 wait complete. 2024-02-21T23:25:44.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.674+0000 7fae454d5640 1 Processor -- start 2024-02-21T23:25:44.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.674+0000 7fae454d5640 1 -- start start 2024-02-21T23:25:44.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.675+0000 7fae454d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae400c6f50 0x7fae400c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:44.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.675+0000 7fae454d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c7c10 0x7fae401f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:44.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.675+0000 7fae454d5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae40154600 0x7fae401f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:44.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.675+0000 7fae454d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400b5720 con 0x7fae400c7c10 2024-02-21T23:25:44.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.675+0000 7fae454d5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400c7780 con 0x7fae40154600 2024-02-21T23:25:44.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.675+0000 7fae454d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae400b58a0 con 0x7fae400c6f50 2024-02-21T23:25:44.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.675+0000 7fae3e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c7c10 0x7fae401f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:44.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.675+0000 7fae3f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae40154600 0x7fae401f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:44.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.676+0000 7fae3effd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae400c6f50 0x7fae400c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:44.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.676+0000 7fae3f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae40154600 0x7fae401f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56206/0 (socket says 172.21.15.133:56206) 2024-02-21T23:25:44.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.676+0000 7fae3f7fe640 1 -- 172.21.15.133:0/3686523580 learned_addr learned my addr 172.21.15.133:0/3686523580 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:44.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.679+0000 7fae3e7fc640 1 -- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae400c6f50 msgr2=0x7fae400c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:44.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.679+0000 7fae3e7fc640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae400c6f50 0x7fae400c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:44.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.679+0000 7fae3e7fc640 1 -- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae40154600 msgr2=0x7fae401f1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:44.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.679+0000 7fae3e7fc640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae40154600 0x7fae401f1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:44.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.679+0000 7fae3e7fc640 1 -- 172.21.15.133:0/3686523580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae401f07e0 con 0x7fae400c7c10 2024-02-21T23:25:44.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.681+0000 7fae3e7fc640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c7c10 0x7fae401f0cc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae400d00a0 tx=0x7fae340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:44.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.685+0000 7fae277fe640 1 -- 172.21.15.133:0/3686523580 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34034040 con 0x7fae400c7c10 2024-02-21T23:25:44.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.685+0000 7fae277fe640 1 -- 172.21.15.133:0/3686523580 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3403ac40 con 0x7fae400c7c10 2024-02-21T23:25:44.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.685+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae401f3f30 con 0x7fae400c7c10 2024-02-21T23:25:44.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.685+0000 7fae277fe640 1 -- 172.21.15.133:0/3686523580 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34044400 con 0x7fae400c7c10 2024-02-21T23:25:44.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.685+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae401f44f0 con 0x7fae400c7c10 2024-02-21T23:25:44.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.686+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4006a130 con 0x7fae400c7c10 2024-02-21T23:25:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.973+0000 7fae277fe640 1 -- 172.21.15.133:0/3686523580 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae34037070 con 0x7fae400c7c10 2024-02-21T23:25:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.974+0000 7fae277fe640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fae2c06b4d0 0x7fae2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:44.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.974+0000 7fae3effd640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fae2c06b4d0 0x7fae2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:44.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.974+0000 7fae3effd640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fae2c06b4d0 0x7fae2c06d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae30005880 tx=0x7fae30009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.988+0000 7fae277fe640 1 -- 172.21.15.133:0/3686523580 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae340305a0 con 0x7fae400c7c10 2024-02-21T23:25:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:44.988+0000 7fae277fe640 1 -- 172.21.15.133:0/3686523580 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fae3404a060 con 0x7fae400c7c10 2024-02-21T23:25:45.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.174+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae400cf3a0 con 0x7fae400c7c10 2024-02-21T23:25:45.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.215+0000 7fae277fe640 1 -- 172.21.15.133:0/3686523580 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fae3403f070 con 0x7fae400c7c10 2024-02-21T23:25:45.202 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:45.203 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:45.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.217+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fae2c06b4d0 msgr2=0x7fae2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.217+0000 7fae454d5640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fae2c06b4d0 0x7fae2c06d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae30005880 tx=0x7fae30009040 comp rx=0 tx=0).stop 2024-02-21T23:25:45.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.217+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c7c10 msgr2=0x7fae401f0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.217+0000 7fae454d5640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c7c10 0x7fae401f0cc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae400d00a0 tx=0x7fae340046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.218+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 shutdown_connections 2024-02-21T23:25:45.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.218+0000 7fae454d5640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fae2c06b4d0 0x7fae2c06d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.218+0000 7fae454d5640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae40154600 0x7fae401f1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.218+0000 7fae454d5640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae400c7c10 0x7fae401f0cc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.218+0000 7fae454d5640 1 --2- 172.21.15.133:0/3686523580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae400c6f50 0x7fae400c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.218+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 >> 172.21.15.133:0/3686523580 conn(0x7fae40062000 msgr2=0x7fae40149b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:45.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.218+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 shutdown_connections 2024-02-21T23:25:45.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.218+0000 7fae454d5640 1 -- 172.21.15.133:0/3686523580 wait complete. 2024-02-21T23:25:45.216 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:45.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.329+0000 7f7276fbb640 1 Processor -- start 2024-02-21T23:25:45.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.329+0000 7f7276fbb640 1 -- start start 2024-02-21T23:25:45.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.330+0000 7f7276fbb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72700cb5f0 0x7f72700cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.330+0000 7f7276fbb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f72700c75e0 0x7f72700c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.330+0000 7f7276fbb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72700c7f30 con 0x7f7270150f80 2024-02-21T23:25:45.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.330+0000 7f7276fbb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72700c2b40 con 0x7f72700c75e0 2024-02-21T23:25:45.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.330+0000 7f7276fbb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72700c2d70 con 0x7f72700cb5f0 2024-02-21T23:25:45.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.331+0000 7f726ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72700cb5f0 0x7f72700cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.331+0000 7f7275558640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f72700c75e0 0x7f72700c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.331+0000 7f726ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72700cb5f0 0x7f72700cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56208/0 (socket says 172.21.15.133:56208) 2024-02-21T23:25:45.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.331+0000 7f726ffff640 1 -- 172.21.15.133:0/16454861 learned_addr learned my addr 172.21.15.133:0/16454861 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:45.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.333+0000 7f726ffff640 1 -- 172.21.15.133:0/16454861 >> v2:172.21.15.133:3300/0 conn(0x7f72700c75e0 msgr2=0x7f72700c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:45.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.333+0000 7f726ffff640 1 --2- 172.21.15.133:0/16454861 >> v2:172.21.15.133:3300/0 conn(0x7f72700c75e0 0x7f72700c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.333+0000 7f726ffff640 1 -- 172.21.15.133:0/16454861 >> v1:172.21.15.133:6789/0 conn(0x7f7270150f80 legacy=0x7f727015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.333+0000 7f726ffff640 1 -- 172.21.15.133:0/16454861 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72700c3050 con 0x7f72700cb5f0 2024-02-21T23:25:45.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.335+0000 7f726ffff640 1 --2- 172.21.15.133:0/16454861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 0x7f72700cba40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7270071680 tx=0x7f726402ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=376584a47639412c server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:45.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.337+0000 7f726f7fe640 1 -- 172.21.15.133:0/16454861 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726402f850 con 0x7f72700cb5f0 2024-02-21T23:25:45.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f726f7fe640 1 -- 172.21.15.133:0/16454861 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726402f9f0 con 0x7f72700cb5f0 2024-02-21T23:25:45.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f726f7fe640 1 -- 172.21.15.133:0/16454861 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726402fdc0 con 0x7f72700cb5f0 2024-02-21T23:25:45.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f7276fbb640 1 -- 172.21.15.133:0/16454861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 msgr2=0x7f72700cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f7276fbb640 1 --2- 172.21.15.133:0/16454861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 0x7f72700cba40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7270071680 tx=0x7f726402ba30 comp rx=0 tx=0).stop 2024-02-21T23:25:45.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f7276fbb640 1 -- 172.21.15.133:0/16454861 shutdown_connections 2024-02-21T23:25:45.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f7276fbb640 1 --2- 172.21.15.133:0/16454861 >> v2:172.21.15.133:3300/0 conn(0x7f72700c75e0 0x7f72700c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f7276fbb640 1 --2- 172.21.15.133:0/16454861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 0x7f72700cba40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f7276fbb640 1 -- 172.21.15.133:0/16454861 >> 172.21.15.133:0/16454861 conn(0x7f727006aec0 msgr2=0x7f727006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:45.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f7276fbb640 1 -- 172.21.15.133:0/16454861 shutdown_connections 2024-02-21T23:25:45.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.339+0000 7f7276fbb640 1 -- 172.21.15.133:0/16454861 wait complete. 2024-02-21T23:25:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.340+0000 7f7276fbb640 1 Processor -- start 2024-02-21T23:25:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.340+0000 7f7276fbb640 1 -- start start 2024-02-21T23:25:45.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.341+0000 7f7276fbb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72700c75e0 0x7f72701627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.341+0000 7f7276fbb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 0x7f727013eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.341+0000 7f7276fbb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7270150f80 0x7f727014f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.341+0000 7f7276fbb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727015a5c0 con 0x7f72700c75e0 2024-02-21T23:25:45.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.341+0000 7f7276fbb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72700c3610 con 0x7f72700cb5f0 2024-02-21T23:25:45.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.341+0000 7f7276fbb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72700c5990 con 0x7f7270150f80 2024-02-21T23:25:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.342+0000 7f7274d57640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72700c75e0 0x7f72701627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.342+0000 7f726ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 0x7f727013eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.342+0000 7f7275558640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7270150f80 0x7f727014f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.342+0000 7f726ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 0x7f727013eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56210/0 (socket says 172.21.15.133:56210) 2024-02-21T23:25:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.342+0000 7f726ffff640 1 -- 172.21.15.133:0/1556303516 learned_addr learned my addr 172.21.15.133:0/1556303516 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:45.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.345+0000 7f7274d57640 1 -- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7270150f80 msgr2=0x7f727014f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.345+0000 7f7274d57640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7270150f80 0x7f727014f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.345+0000 7f7274d57640 1 -- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 msgr2=0x7f727013eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.345+0000 7f7274d57640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 0x7f727013eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.345+0000 7f7274d57640 1 -- 172.21.15.133:0/1556303516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72701f3230 con 0x7f72700c75e0 2024-02-21T23:25:45.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.347+0000 7f7274d57640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72700c75e0 0x7f72701627c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f726000c3f0 tx=0x7f726000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:45.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.351+0000 7f726d7fa640 1 -- 172.21.15.133:0/1556303516 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600052d0 con 0x7f72700c75e0 2024-02-21T23:25:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.351+0000 7f726d7fa640 1 -- 172.21.15.133:0/1556303516 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260005490 con 0x7f72700c75e0 2024-02-21T23:25:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.351+0000 7f726d7fa640 1 -- 172.21.15.133:0/1556303516 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260007260 con 0x7f72700c75e0 2024-02-21T23:25:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.351+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727013d1b0 con 0x7f72700c75e0 2024-02-21T23:25:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.351+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72701f34d0 con 0x7f72700c75e0 2024-02-21T23:25:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.352+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72700cc4b0 con 0x7f72700c75e0 2024-02-21T23:25:45.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.369+0000 7f726d7fa640 1 -- 172.21.15.133:0/1556303516 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7260007400 con 0x7f72700c75e0 2024-02-21T23:25:45.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.369+0000 7f726d7fa640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f725c06b4d0 0x7f725c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.369+0000 7f726ffff640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f725c06b4d0 0x7f725c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.370+0000 7f726ffff640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f725c06b4d0 0x7f725c06d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f72700c6ac0 tx=0x7f7264008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.372+0000 7f726d7fa640 1 -- 172.21.15.133:0/1556303516 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f72600566a0 con 0x7f72700c75e0 2024-02-21T23:25:45.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.606+0000 7f726d7fa640 1 -- 172.21.15.133:0/1556303516 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f72700cc4b0 con 0x7f72700c75e0 2024-02-21T23:25:45.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.783+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f727006a130 con 0x7f72700c75e0 2024-02-21T23:25:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.821+0000 7f726d7fa640 1 -- 172.21.15.133:0/1556303516 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f7260058600 con 0x7f72700c75e0 2024-02-21T23:25:45.809 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:45.809 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f725c06b4d0 msgr2=0x7f725c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f725c06b4d0 0x7f725c06d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f72700c6ac0 tx=0x7f7264008030 comp rx=0 tx=0).stop 2024-02-21T23:25:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72700c75e0 msgr2=0x7f72701627c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72700c75e0 0x7f72701627c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f726000c3f0 tx=0x7f726000c8b0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 shutdown_connections 2024-02-21T23:25:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f725c06b4d0 0x7f725c06d990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7270150f80 0x7f727014f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72700cb5f0 0x7f727013eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 --2- 172.21.15.133:0/1556303516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72700c75e0 0x7f72701627c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 >> 172.21.15.133:0/1556303516 conn(0x7f727006aec0 msgr2=0x7f72700cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 shutdown_connections 2024-02-21T23:25:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.824+0000 7f7276fbb640 1 -- 172.21.15.133:0/1556303516 wait complete. 2024-02-21T23:25:45.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-21T23:25:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.940+0000 7f3ee391e640 1 Processor -- start 2024-02-21T23:25:45.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.940+0000 7f3ee391e640 1 -- start start 2024-02-21T23:25:45.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.941+0000 7f3ee391e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3edc15aa30 0x7f3edc15da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.941+0000 7f3ee391e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc15df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.941+0000 7f3ee391e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3edc0c9d20 con 0x7f3edc1540d0 2024-02-21T23:25:45.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.941+0000 7f3ee391e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc15d5a0 con 0x7f3edc15aa30 2024-02-21T23:25:45.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.941+0000 7f3ee391e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc0c96c0 con 0x7f3edc0c8940 2024-02-21T23:25:45.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.941+0000 7f3ee1ebb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.942+0000 7f3ee0eb9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3edc15aa30 0x7f3edc15da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.942+0000 7f3ee16ba640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f3edc1540d0 0x7f3edc0cd900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:36078/0 (socket says 172.21.15.133:36078) 2024-02-21T23:25:45.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.942+0000 7f3ee16ba640 1 -- 172.21.15.133:0/3970156099 learned_addr learned my addr 172.21.15.133:0/3970156099 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:45.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.944+0000 7f3ee1ebb640 1 -- 172.21.15.133:0/3970156099 >> v2:172.21.15.142:3300/0 conn(0x7f3edc15aa30 msgr2=0x7f3edc15da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:45.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.944+0000 7f3ee1ebb640 1 --2- 172.21.15.133:0/3970156099 >> v2:172.21.15.142:3300/0 conn(0x7f3edc15aa30 0x7f3edc15da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.944+0000 7f3ee1ebb640 1 -- 172.21.15.133:0/3970156099 >> v1:172.21.15.142:6789/0 conn(0x7f3edc1540d0 legacy=0x7f3edc0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.944+0000 7f3ee1ebb640 1 -- 172.21.15.133:0/3970156099 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc0c9fc0 con 0x7f3edc0c8940 2024-02-21T23:25:45.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.946+0000 7f3ee1ebb640 1 --2- 172.21.15.133:0/3970156099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc15df70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009d90 tx=0x7f3ed402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbfc656afedea4f server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:45.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.948+0000 7f3ed3fff640 1 -- 172.21.15.133:0/3970156099 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4039070 con 0x7f3edc0c8940 2024-02-21T23:25:45.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.950+0000 7f3ed3fff640 1 -- 172.21.15.133:0/3970156099 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed402cbc0 con 0x7f3edc0c8940 2024-02-21T23:25:45.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.950+0000 7f3ed3fff640 1 -- 172.21.15.133:0/3970156099 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4034440 con 0x7f3edc0c8940 2024-02-21T23:25:45.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.951+0000 7f3ee391e640 1 -- 172.21.15.133:0/3970156099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 msgr2=0x7f3edc15df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.951+0000 7f3ee391e640 1 --2- 172.21.15.133:0/3970156099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc15df70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009d90 tx=0x7f3ed402be10 comp rx=0 tx=0).stop 2024-02-21T23:25:45.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.951+0000 7f3ee391e640 1 -- 172.21.15.133:0/3970156099 shutdown_connections 2024-02-21T23:25:45.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.951+0000 7f3ee391e640 1 --2- 172.21.15.133:0/3970156099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc15df70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.951+0000 7f3ee391e640 1 --2- 172.21.15.133:0/3970156099 >> v2:172.21.15.142:3300/0 conn(0x7f3edc15aa30 0x7f3edc15da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.951+0000 7f3ee391e640 1 -- 172.21.15.133:0/3970156099 >> 172.21.15.133:0/3970156099 conn(0x7f3edc06f7f0 msgr2=0x7f3edc061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.951+0000 7f3ee391e640 1 -- 172.21.15.133:0/3970156099 shutdown_connections 2024-02-21T23:25:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.951+0000 7f3ee391e640 1 -- 172.21.15.133:0/3970156099 wait complete. 2024-02-21T23:25:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.952+0000 7f3ee391e640 1 Processor -- start 2024-02-21T23:25:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.952+0000 7f3ee391e640 1 -- start start 2024-02-21T23:25:45.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.953+0000 7f3ee391e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc1f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.953+0000 7f3ee391e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3edc1540d0 0x7f3edc14eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.953+0000 7f3ee391e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc15aa30 0x7f3edc14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:45.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.953+0000 7f3ee391e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc0ce010 con 0x7f3edc15aa30 2024-02-21T23:25:45.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.953+0000 7f3ee391e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc06c040 con 0x7f3edc1540d0 2024-02-21T23:25:45.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.953+0000 7f3ee391e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc0ce190 con 0x7f3edc0c8940 2024-02-21T23:25:45.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.954+0000 7f3ee16ba640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc1f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.954+0000 7f3ee1ebb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc15aa30 0x7f3edc14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.954+0000 7f3ee0eb9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3edc1540d0 0x7f3edc14eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:45.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.954+0000 7f3ee16ba640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc1f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:51730/0 (socket says 172.21.15.133:51730) 2024-02-21T23:25:45.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.954+0000 7f3ee16ba640 1 -- 172.21.15.133:0/654185069 learned_addr learned my addr 172.21.15.133:0/654185069 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:45.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.957+0000 7f3ee16ba640 1 -- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3edc1540d0 msgr2=0x7f3edc14eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.957+0000 7f3ee16ba640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3edc1540d0 0x7f3edc14eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.957+0000 7f3ee16ba640 1 -- 172.21.15.133:0/654185069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc15aa30 msgr2=0x7f3edc14d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:45.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.957+0000 7f3ee16ba640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc15aa30 0x7f3edc14d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:45.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.957+0000 7f3ee16ba640 1 -- 172.21.15.133:0/654185069 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc1521f0 con 0x7f3edc0c8940 2024-02-21T23:25:45.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.959+0000 7f3ee16ba640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc1f3ce0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800c400 tx=0x7f3ed800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:45.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.963+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/654185069 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8017070 con 0x7f3edc0c8940 2024-02-21T23:25:45.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.963+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/654185069 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed8005ce0 con 0x7f3edc0c8940 2024-02-21T23:25:45.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.963+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc1f4220 con 0x7f3edc0c8940 2024-02-21T23:25:45.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.963+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/654185069 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80072f0 con 0x7f3edc0c8940 2024-02-21T23:25:45.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.963+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc0cdb70 con 0x7f3edc0c8940 2024-02-21T23:25:45.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:45.964+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc06a130 con 0x7f3edc0c8940 2024-02-21T23:25:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:46.218+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/654185069 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ed801a070 con 0x7f3edc0c8940 2024-02-21T23:25:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:46.218+0000 7f3ed1ffb640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ec006b520 0x7f3ec006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:46.218+0000 7f3ee0eb9640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ec006b520 0x7f3ec006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:46.219+0000 7f3ee0eb9640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ec006b520 0x7f3ec006d9e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0c8690 tx=0x7f3ecc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:46.224+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/654185069 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ed80562a0 con 0x7f3edc0c8940 2024-02-21T23:25:46.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:46.237+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/654185069 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f3ed8021020 con 0x7f3edc0c8940 2024-02-21T23:25:46.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:46.449+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3edc0c2990 con 0x7f3edc0c8940 2024-02-21T23:25:47.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.435+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/654185069 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3ed8058200 con 0x7f3edc0c8940 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ec006b520 msgr2=0x7f3ec006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ec006b520 0x7f3ec006d9e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0c8690 tx=0x7f3ecc009190 comp rx=0 tx=0).stop 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 msgr2=0x7f3edc1f3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc1f3ce0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800c400 tx=0x7f3ed800c8c0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 shutdown_connections 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ec006b520 0x7f3ec006d9e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc15aa30 0x7f3edc14d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3edc1540d0 0x7f3edc14eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 --2- 172.21.15.133:0/654185069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3edc0c8940 0x7f3edc1f3ce0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 >> 172.21.15.133:0/654185069 conn(0x7f3edc06f7f0 msgr2=0x7f3edc1588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 shutdown_connections 2024-02-21T23:25:47.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.437+0000 7f3ee391e640 1 -- 172.21.15.133:0/654185069 wait complete. 2024-02-21T23:25:47.434 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300} 2024-02-21T23:25:47.434 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-21T23:25:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.551+0000 7f3f04555640 1 Processor -- start 2024-02-21T23:25:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.551+0000 7f3f04555640 1 -- start start 2024-02-21T23:25:47.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.552+0000 7f3f04555640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc154600 0x7f3efc0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:47.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.552+0000 7f3f04555640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3efc0c6f50 0x7f3efc0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:47.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.552+0000 7f3f04555640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc15ab40 con 0x7f3efc0cab80 2024-02-21T23:25:47.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.552+0000 7f3f04555640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3efc0c74a0 con 0x7f3efc0c6f50 2024-02-21T23:25:47.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.552+0000 7f3f04555640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0c7780 con 0x7f3efc154600 2024-02-21T23:25:47.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.553+0000 7f3f01af0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3efc0c6f50 0x7f3efc0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:47.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.553+0000 7f3f02af2640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3efc0cab80 0x7f3efc0c7c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60430/0 (socket says 172.21.15.133:60430) 2024-02-21T23:25:47.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.553+0000 7f3f02af2640 1 -- 172.21.15.133:0/1977640267 learned_addr learned my addr 172.21.15.133:0/1977640267 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:47.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.553+0000 7f3f022f1640 1 --2- 172.21.15.133:0/1977640267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc154600 0x7f3efc0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:47.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.555+0000 7f3f012ef640 1 -- 172.21.15.133:0/1977640267 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354535495 0 0) 0x7f3efc15ab40 con 0x7f3efc0cab80 2024-02-21T23:25:47.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.555+0000 7f3f012ef640 1 -- 172.21.15.133:0/1977640267 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee8003520 con 0x7f3efc0cab80 2024-02-21T23:25:47.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.555+0000 7f3f01af0640 1 -- 172.21.15.133:0/1977640267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc154600 msgr2=0x7f3efc0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:47.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.555+0000 7f3f01af0640 1 --2- 172.21.15.133:0/1977640267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc154600 0x7f3efc0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.555+0000 7f3f01af0640 1 -- 172.21.15.133:0/1977640267 >> v1:172.21.15.133:6789/0 conn(0x7f3efc0cab80 legacy=0x7f3efc0c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:47.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.555+0000 7f3f01af0640 1 -- 172.21.15.133:0/1977640267 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc15ae00 con 0x7f3efc0c6f50 2024-02-21T23:25:47.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.557+0000 7f3f01af0640 1 --2- 172.21.15.133:0/1977640267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc0c6f50 0x7f3efc0ca640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0097a0 tx=0x7f3eec02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b30d9b27b86fb4b server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:47.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.561+0000 7f3f012ef640 1 -- 172.21.15.133:0/1977640267 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec035020 con 0x7f3efc0c6f50 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.561+0000 7f3f012ef640 1 -- 172.21.15.133:0/1977640267 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec031830 con 0x7f3efc0c6f50 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.561+0000 7f3f012ef640 1 -- 172.21.15.133:0/1977640267 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec031c00 con 0x7f3efc0c6f50 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.562+0000 7f3f04555640 1 -- 172.21.15.133:0/1977640267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc0c6f50 msgr2=0x7f3efc0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.562+0000 7f3f04555640 1 --2- 172.21.15.133:0/1977640267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc0c6f50 0x7f3efc0ca640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0097a0 tx=0x7f3eec02b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.562+0000 7f3f04555640 1 -- 172.21.15.133:0/1977640267 shutdown_connections 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.562+0000 7f3f04555640 1 --2- 172.21.15.133:0/1977640267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc0c6f50 0x7f3efc0ca640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.562+0000 7f3f04555640 1 --2- 172.21.15.133:0/1977640267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc154600 0x7f3efc0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.562+0000 7f3f04555640 1 -- 172.21.15.133:0/1977640267 >> 172.21.15.133:0/1977640267 conn(0x7f3efc062000 msgr2=0x7f3efc0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.562+0000 7f3f04555640 1 -- 172.21.15.133:0/1977640267 shutdown_connections 2024-02-21T23:25:47.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.562+0000 7f3f04555640 1 -- 172.21.15.133:0/1977640267 wait complete. 2024-02-21T23:25:47.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.563+0000 7f3f04555640 1 Processor -- start 2024-02-21T23:25:47.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.563+0000 7f3f04555640 1 -- start start 2024-02-21T23:25:47.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f04555640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0c6f50 0x7f3efc1efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:47.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f04555640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc0cab80 0x7f3efc157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:47.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f04555640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc154600 0x7f3efc149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:47.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f04555640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0b5720 con 0x7f3efc0cab80 2024-02-21T23:25:47.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f04555640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0cb860 con 0x7f3efc154600 2024-02-21T23:25:47.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f04555640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0b58a0 con 0x7f3efc0c6f50 2024-02-21T23:25:47.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f02af2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc154600 0x7f3efc149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:47.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f022f1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0c6f50 0x7f3efc1efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:47.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f01af0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc0cab80 0x7f3efc157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:47.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f02af2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc154600 0x7f3efc149f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56240/0 (socket says 172.21.15.133:56240) 2024-02-21T23:25:47.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.564+0000 7f3f02af2640 1 -- 172.21.15.133:0/1189559982 learned_addr learned my addr 172.21.15.133:0/1189559982 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:47.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.568+0000 7f3f02af2640 1 -- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0c6f50 msgr2=0x7f3efc1efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:47.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.568+0000 7f3f02af2640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0c6f50 0x7f3efc1efe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.568+0000 7f3f02af2640 1 -- 172.21.15.133:0/1189559982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc0cab80 msgr2=0x7f3efc157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:47.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.568+0000 7f3f02af2640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc0cab80 0x7f3efc157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:47.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.568+0000 7f3f02af2640 1 -- 172.21.15.133:0/1189559982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc157a70 con 0x7f3efc154600 2024-02-21T23:25:47.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.568+0000 7f3f022f1640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0c6f50 0x7f3efc1efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:25:47.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.570+0000 7f3f02af2640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc154600 0x7f3efc149f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400c460 tx=0x7f3ef400c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:47.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.572+0000 7f3ef2ffd640 1 -- 172.21.15.133:0/1189559982 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4017070 con 0x7f3efc154600 2024-02-21T23:25:47.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.573+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc1f0450 con 0x7f3efc154600 2024-02-21T23:25:47.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.573+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc1f09f0 con 0x7f3efc154600 2024-02-21T23:25:47.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.574+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc06a130 con 0x7f3efc154600 2024-02-21T23:25:47.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.823+0000 7f3ef2ffd640 1 -- 172.21.15.133:0/1189559982 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef4005ce0 con 0x7f3efc154600 2024-02-21T23:25:47.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.823+0000 7f3ef2ffd640 1 -- 172.21.15.133:0/1189559982 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4021060 con 0x7f3efc154600 2024-02-21T23:25:47.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.823+0000 7f3ef2ffd640 1 -- 172.21.15.133:0/1189559982 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ef401a070 con 0x7f3efc154600 2024-02-21T23:25:47.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.823+0000 7f3ef2ffd640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ee006b4d0 0x7f3ee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:47.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.824+0000 7f3f022f1640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ee006b4d0 0x7f3ee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:47.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.824+0000 7f3f022f1640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ee006b4d0 0x7f3ee006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009fd0 tx=0x7f3ef800b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:47.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.826+0000 7f3ef2ffd640 1 -- 172.21.15.133:0/1189559982 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ef4056640 con 0x7f3efc154600 2024-02-21T23:25:47.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:47.836+0000 7f3ef2ffd640 1 -- 172.21.15.133:0/1189559982 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f3ef40585a0 con 0x7f3efc154600 2024-02-21T23:25:48.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.044+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3efc14d8c0 con 0x7f3efc154600 2024-02-21T23:25:48.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.515+0000 7f3ef2ffd640 1 -- 172.21.15.133:0/1189559982 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3ef405c640 con 0x7f3efc154600 2024-02-21T23:25:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ee006b4d0 msgr2=0x7f3ee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:48.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ee006b4d0 0x7f3ee006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009fd0 tx=0x7f3ef800b040 comp rx=0 tx=0).stop 2024-02-21T23:25:48.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc154600 msgr2=0x7f3efc149f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:48.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc154600 0x7f3efc149f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400c460 tx=0x7f3ef400c920 comp rx=0 tx=0).stop 2024-02-21T23:25:48.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 shutdown_connections 2024-02-21T23:25:48.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3ee006b4d0 0x7f3ee006d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:48.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc154600 0x7f3efc149f80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:48.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc0cab80 0x7f3efc157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:48.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 --2- 172.21.15.133:0/1189559982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0c6f50 0x7f3efc1efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 >> 172.21.15.133:0/1189559982 conn(0x7f3efc062000 msgr2=0x7f3efc0cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 shutdown_connections 2024-02-21T23:25:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.517+0000 7f3f04555640 1 -- 172.21.15.133:0/1189559982 wait complete. 2024-02-21T23:25:48.515 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-21T23:25:48.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.633+0000 7f940c51e640 1 Processor -- start 2024-02-21T23:25:48.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.633+0000 7f940c51e640 1 -- start start 2024-02-21T23:25:48.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.633+0000 7f940c51e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f940415aa30 0x7f940415da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:48.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.633+0000 7f940c51e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94040c9d20 con 0x7f94040c8940 2024-02-21T23:25:48.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.634+0000 7f940c51e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94040c96c0 con 0x7f94041540d0 2024-02-21T23:25:48.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.634+0000 7f940c51e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940415d5a0 con 0x7f940415aa30 2024-02-21T23:25:48.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.634+0000 7f940a2ba640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f94041540d0 0x7f94040cd900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:36090/0 (socket says 172.21.15.133:36090) 2024-02-21T23:25:48.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.634+0000 7f940a2ba640 1 -- 172.21.15.133:0/3843615739 learned_addr learned my addr 172.21.15.133:0/3843615739 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:48.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.634+0000 7f9409ab9640 1 --2- 172.21.15.133:0/3843615739 >> v2:172.21.15.133:3300/0 conn(0x7f940415aa30 0x7f940415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:48.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.636+0000 7f94092b8640 1 -- 172.21.15.133:0/3843615739 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785625036 0 0) 0x7f94040c96c0 con 0x7f94041540d0 2024-02-21T23:25:48.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.636+0000 7f94092b8640 1 -- 172.21.15.133:0/3843615739 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f0003520 con 0x7f94041540d0 2024-02-21T23:25:48.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.637+0000 7f9409ab9640 1 -- 172.21.15.133:0/3843615739 >> v1:172.21.15.142:6789/0 conn(0x7f94041540d0 legacy=0x7f94040cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:48.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.637+0000 7f9409ab9640 1 -- 172.21.15.133:0/3843615739 >> v1:172.21.15.133:6789/0 conn(0x7f94040c8940 legacy=0x7f9404164930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:48.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.637+0000 7f9409ab9640 1 -- 172.21.15.133:0/3843615739 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94040c9fc0 con 0x7f940415aa30 2024-02-21T23:25:48.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.640+0000 7f9409ab9640 1 --2- 172.21.15.133:0/3843615739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f940415aa30 0x7f940415da30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f93f40097a0 tx=0x7f93f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8a0dfc31135f16 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:48.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.645+0000 7f94092b8640 1 -- 172.21.15.133:0/3843615739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4035020 con 0x7f940415aa30 2024-02-21T23:25:48.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.645+0000 7f94092b8640 1 -- 172.21.15.133:0/3843615739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f4031830 con 0x7f940415aa30 2024-02-21T23:25:48.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.645+0000 7f94092b8640 1 -- 172.21.15.133:0/3843615739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4031c00 con 0x7f940415aa30 2024-02-21T23:25:48.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.646+0000 7f940c51e640 1 -- 172.21.15.133:0/3843615739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f940415aa30 msgr2=0x7f940415da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:48.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.646+0000 7f940c51e640 1 --2- 172.21.15.133:0/3843615739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f940415aa30 0x7f940415da30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f93f40097a0 tx=0x7f93f402b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:48.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.646+0000 7f940c51e640 1 -- 172.21.15.133:0/3843615739 shutdown_connections 2024-02-21T23:25:48.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.646+0000 7f940c51e640 1 --2- 172.21.15.133:0/3843615739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f940415aa30 0x7f940415da30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:48.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.646+0000 7f940c51e640 1 -- 172.21.15.133:0/3843615739 >> 172.21.15.133:0/3843615739 conn(0x7f940406f7f0 msgr2=0x7f9404061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:48.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.646+0000 7f940c51e640 1 -- 172.21.15.133:0/3843615739 shutdown_connections 2024-02-21T23:25:48.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.646+0000 7f940c51e640 1 -- 172.21.15.133:0/3843615739 wait complete. 2024-02-21T23:25:48.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.647+0000 7f940c51e640 1 Processor -- start 2024-02-21T23:25:48.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.647+0000 7f940c51e640 1 -- start start 2024-02-21T23:25:48.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.648+0000 7f940c51e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94040c8940 0x7f94040cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:48.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.648+0000 7f940c51e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94041540d0 0x7f94041efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:48.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.648+0000 7f940c51e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415aa30 0x7f94041f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:48.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.648+0000 7f940c51e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94040ce010 con 0x7f94040c8940 2024-02-21T23:25:48.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.648+0000 7f940c51e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940406c040 con 0x7f940415aa30 2024-02-21T23:25:48.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.648+0000 7f940c51e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94040ce190 con 0x7f94041540d0 2024-02-21T23:25:48.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.648+0000 7f940a2ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94040c8940 0x7f94040cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:48.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.648+0000 7f940aabb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415aa30 0x7f94041f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:48.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.648+0000 7f9409ab9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94041540d0 0x7f94041efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:48.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.649+0000 7f940a2ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94040c8940 0x7f94040cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55972/0 (socket says 172.21.15.133:55972) 2024-02-21T23:25:48.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.649+0000 7f940a2ba640 1 -- 172.21.15.133:0/1275991361 learned_addr learned my addr 172.21.15.133:0/1275991361 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:48.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.652+0000 7f940a2ba640 1 -- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94041540d0 msgr2=0x7f94041efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:48.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.652+0000 7f940a2ba640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94041540d0 0x7f94041efe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:48.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.652+0000 7f940a2ba640 1 -- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415aa30 msgr2=0x7f94041f03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:48.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.652+0000 7f940a2ba640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415aa30 0x7f94041f03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:48.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.652+0000 7f940a2ba640 1 -- 172.21.15.133:0/1275991361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94041504f0 con 0x7f94040c8940 2024-02-21T23:25:48.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.654+0000 7f940a2ba640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94040c8940 0x7f94040cca80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f940000cb40 tx=0x7f940000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.658+0000 7f93faffd640 1 -- 172.21.15.133:0/1275991361 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400017070 con 0x7f94040c8940 2024-02-21T23:25:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.658+0000 7f93faffd640 1 -- 172.21.15.133:0/1275991361 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940000aca0 con 0x7f94040c8940 2024-02-21T23:25:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.658+0000 7f93faffd640 1 -- 172.21.15.133:0/1275991361 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400012620 con 0x7f94040c8940 2024-02-21T23:25:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.658+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940415d270 con 0x7f94040c8940 2024-02-21T23:25:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.658+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9404150760 con 0x7f94040c8940 2024-02-21T23:25:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.659+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940406db10 con 0x7f94040c8940 2024-02-21T23:25:48.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.677+0000 7f93faffd640 1 -- 172.21.15.133:0/1275991361 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f940001a070 con 0x7f94040c8940 2024-02-21T23:25:48.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.678+0000 7f93faffd640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f93e806b520 0x7f93e806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:48.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.678+0000 7f9409ab9640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f93e806b520 0x7f93e806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:48.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.678+0000 7f9409ab9640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f93e806b520 0x7f93e806d9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93f4036000 tx=0x7f93f403c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:48.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.921+0000 7f93faffd640 1 -- 172.21.15.133:0/1275991361 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9400056d60 con 0x7f94040c8940 2024-02-21T23:25:48.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:48.921+0000 7f93faffd640 1 -- 172.21.15.133:0/1275991361 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f940406db10 con 0x7f94040c8940 2024-02-21T23:25:49.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.134+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f940406a130 con 0x7f94040c8940 2024-02-21T23:25:49.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.538+0000 7f93faffd640 1 -- 172.21.15.133:0/1275991361 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v10) v1 ==== 115+0+0 (secure 0 0 0) 0x7f9400021020 con 0x7f94040c8940 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f93e806b520 msgr2=0x7f93e806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f93e806b520 0x7f93e806d9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93f4036000 tx=0x7f93f403c040 comp rx=0 tx=0).stop 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94040c8940 msgr2=0x7f94040cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94040c8940 0x7f94040cca80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f940000cb40 tx=0x7f940000a060 comp rx=0 tx=0).stop 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 shutdown_connections 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f93e806b520 0x7f93e806d9e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f940415aa30 0x7f94041f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94041540d0 0x7f94041efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 --2- 172.21.15.133:0/1275991361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94040c8940 0x7f94040cca80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 >> 172.21.15.133:0/1275991361 conn(0x7f940406f7f0 msgr2=0x7f9404061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:49.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 shutdown_connections 2024-02-21T23:25:49.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.540+0000 7f940c51e640 1 -- 172.21.15.133:0/1275991361 wait complete. 2024-02-21T23:25:49.538 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-21T23:25:49.538 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:25:49.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.653+0000 7fa7a3a95640 1 Processor -- start 2024-02-21T23:25:49.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.653+0000 7fa7a3a95640 1 -- start start 2024-02-21T23:25:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.654+0000 7fa7a3a95640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa79c0c6cc0 0x7fa79c0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.654+0000 7fa7a3a95640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0cc500 0x7fa79c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.654+0000 7fa7a3a95640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79c15a0c0 con 0x7fa79c154490 2024-02-21T23:25:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.654+0000 7fa7a3a95640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa79c154870 con 0x7fa79c0c6cc0 2024-02-21T23:25:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.654+0000 7fa7a3a95640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0d03e0 con 0x7fa79c0cc500 2024-02-21T23:25:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.655+0000 7fa7a1831640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fa79c154490 0x7fa79c0ca720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:36106/0 (socket says 172.21.15.133:36106) 2024-02-21T23:25:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.655+0000 7fa7a1831640 1 -- 172.21.15.133:0/3652035214 learned_addr learned my addr 172.21.15.133:0/3652035214 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.655+0000 7fa7a2032640 1 --2- 172.21.15.133:0/3652035214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0cc500 0x7fa79c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.655+0000 7fa7a1030640 1 --2- 172.21.15.133:0/3652035214 >> v2:172.21.15.133:3300/0 conn(0x7fa79c0c6cc0 0x7fa79c0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:49.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.657+0000 7fa7a082f640 1 -- 172.21.15.133:0/3652035214 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330214495 0 0) 0x7fa79c15a0c0 con 0x7fa79c154490 2024-02-21T23:25:49.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.657+0000 7fa7a082f640 1 -- 172.21.15.133:0/3652035214 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa788003520 con 0x7fa79c154490 2024-02-21T23:25:49.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.657+0000 7fa7a1030640 1 -- 172.21.15.133:0/3652035214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0cc500 msgr2=0x7fa79c0cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:49.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.657+0000 7fa7a1030640 1 --2- 172.21.15.133:0/3652035214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0cc500 0x7fa79c0cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.657+0000 7fa7a1030640 1 -- 172.21.15.133:0/3652035214 >> v1:172.21.15.142:6789/0 conn(0x7fa79c154490 legacy=0x7fa79c0ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:49.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.657+0000 7fa7a1030640 1 -- 172.21.15.133:0/3652035214 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c15a380 con 0x7fa79c0c6cc0 2024-02-21T23:25:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.660+0000 7fa7a1030640 1 --2- 172.21.15.133:0/3652035214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c0c6cc0 0x7fa79c0cbfc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009790 tx=0x7fa78c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5863350f50d5a63e server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:49.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.663+0000 7fa7a082f640 1 -- 172.21.15.133:0/3652035214 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c035020 con 0x7fa79c0c6cc0 2024-02-21T23:25:49.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.663+0000 7fa7a082f640 1 -- 172.21.15.133:0/3652035214 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c031800 con 0x7fa79c0c6cc0 2024-02-21T23:25:49.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.663+0000 7fa7a082f640 1 -- 172.21.15.133:0/3652035214 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c031bd0 con 0x7fa79c0c6cc0 2024-02-21T23:25:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.664+0000 7fa7a3a95640 1 -- 172.21.15.133:0/3652035214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c0c6cc0 msgr2=0x7fa79c0cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.664+0000 7fa7a3a95640 1 --2- 172.21.15.133:0/3652035214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c0c6cc0 0x7fa79c0cbfc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009790 tx=0x7fa78c02b8a0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.664+0000 7fa7a3a95640 1 -- 172.21.15.133:0/3652035214 shutdown_connections 2024-02-21T23:25:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.664+0000 7fa7a3a95640 1 --2- 172.21.15.133:0/3652035214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0cc500 0x7fa79c0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.664+0000 7fa7a3a95640 1 --2- 172.21.15.133:0/3652035214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c0c6cc0 0x7fa79c0cbfc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.664+0000 7fa7a3a95640 1 -- 172.21.15.133:0/3652035214 >> 172.21.15.133:0/3652035214 conn(0x7fa79c072800 msgr2=0x7fa79c06be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:49.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.664+0000 7fa7a3a95640 1 -- 172.21.15.133:0/3652035214 shutdown_connections 2024-02-21T23:25:49.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.664+0000 7fa7a3a95640 1 -- 172.21.15.133:0/3652035214 wait complete. 2024-02-21T23:25:49.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.665+0000 7fa7a3a95640 1 Processor -- start 2024-02-21T23:25:49.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.665+0000 7fa7a3a95640 1 -- start start 2024-02-21T23:25:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.666+0000 7fa7a3a95640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0c6cc0 0x7fa79c158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.666+0000 7fa7a3a95640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79c0cc500 0x7fa79c0ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.666+0000 7fa7a3a95640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c154490 0x7fa79c14f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.666+0000 7fa7a3a95640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c15ac20 con 0x7fa79c154490 2024-02-21T23:25:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.666+0000 7fa7a3a95640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c15a940 con 0x7fa79c0cc500 2024-02-21T23:25:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.666+0000 7fa7a3a95640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0cae30 con 0x7fa79c0c6cc0 2024-02-21T23:25:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.667+0000 7fa7a2032640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c154490 0x7fa79c14f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.667+0000 7fa7a1030640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79c0cc500 0x7fa79c0ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.667+0000 7fa7a1831640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0c6cc0 0x7fa79c158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.667+0000 7fa7a1030640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79c0cc500 0x7fa79c0ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56254/0 (socket says 172.21.15.133:56254) 2024-02-21T23:25:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.667+0000 7fa7a1030640 1 -- 172.21.15.133:0/1067803059 learned_addr learned my addr 172.21.15.133:0/1067803059 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.670+0000 7fa7a1030640 1 -- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0c6cc0 msgr2=0x7fa79c158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.670+0000 7fa7a1030640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0c6cc0 0x7fa79c158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.670+0000 7fa7a1030640 1 -- 172.21.15.133:0/1067803059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c154490 msgr2=0x7fa79c14f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.670+0000 7fa7a1030640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c154490 0x7fa79c14f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.670+0000 7fa7a1030640 1 -- 172.21.15.133:0/1067803059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c0cf660 con 0x7fa79c0cc500 2024-02-21T23:25:49.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.672+0000 7fa7a1030640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79c0cc500 0x7fa79c0ce0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa78c034040 tx=0x7fa78c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:49.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.676+0000 7fa7927fc640 1 -- 172.21.15.133:0/1067803059 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c040070 con 0x7fa79c0cc500 2024-02-21T23:25:49.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.676+0000 7fa7927fc640 1 -- 172.21.15.133:0/1067803059 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c03e070 con 0x7fa79c0cc500 2024-02-21T23:25:49.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.676+0000 7fa7927fc640 1 -- 172.21.15.133:0/1067803059 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c035040 con 0x7fa79c0cc500 2024-02-21T23:25:49.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.676+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c0c49c0 con 0x7fa79c0cc500 2024-02-21T23:25:49.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.676+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c0cf380 con 0x7fa79c0cc500 2024-02-21T23:25:49.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.677+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79c06a130 con 0x7fa79c0cc500 2024-02-21T23:25:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.696+0000 7fa7927fc640 1 -- 172.21.15.133:0/1067803059 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa78c038020 con 0x7fa79c0cc500 2024-02-21T23:25:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.696+0000 7fa7927fc640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa78006b550 0x7fa78006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.696+0000 7fa7a1831640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa78006b550 0x7fa78006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.697+0000 7fa7a1831640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa78006b550 0x7fa78006da10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa798005c00 tx=0x7fa798009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:49.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.927+0000 7fa7927fc640 1 -- 172.21.15.133:0/1067803059 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa78c078710 con 0x7fa79c0cc500 2024-02-21T23:25:49.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:49.938+0000 7fa7927fc640 1 -- 172.21.15.133:0/1067803059 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fa78c07a670 con 0x7fa79c0cc500 2024-02-21T23:25:50.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.143+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa79c0cbda0 con 0x7fa79c0cc500 2024-02-21T23:25:50.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.168+0000 7fa7927fc640 1 -- 172.21.15.133:0/1067803059 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6535 (secure 0 0 0) 0x7fa78c07e710 con 0x7fa79c0cc500 2024-02-21T23:25:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:50.155 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1319340142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1319340142},{"type":"v1","addr":"172.21.15.133:6837","nonce":1319340142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1740931674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1740931674},{"type":"v1","addr":"172.21.15.142:6837","nonce":1740931674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3712864993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3712864993},{"type":"v1","addr":"172.21.15.142:6839","nonce":3712864993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2928905924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2928905924},{"type":"v1","addr":"172.21.15.133:6833","nonce":2928905924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3461010406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3461010406},{"type":"v1","addr":"172.21.15.133:6835","nonce":3461010406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-21T23:25:16.775942+0000","modified":"2024-02-21T23:25:49.141553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6835/1527124199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1527124199},{"type":"v1","addr":"172.21.15.142:6835","nonce":1527124199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-21T23:25:50.155 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2024-02-21T23:25:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa78006b550 msgr2=0x7fa78006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa78006b550 0x7fa78006da10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa798005c00 tx=0x7fa798009040 comp rx=0 tx=0).stop 2024-02-21T23:25:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79c0cc500 msgr2=0x7fa79c0ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79c0cc500 0x7fa79c0ce0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa78c034040 tx=0x7fa78c0046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 shutdown_connections 2024-02-21T23:25:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa78006b550 0x7fa78006da10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c154490 0x7fa79c14f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79c0cc500 0x7fa79c0ce0f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 --2- 172.21.15.133:0/1067803059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa79c0c6cc0 0x7fa79c158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 >> 172.21.15.133:0/1067803059 conn(0x7fa79c072800 msgr2=0x7fa79c0c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 shutdown_connections 2024-02-21T23:25:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.170+0000 7fa7a3a95640 1 -- 172.21.15.133:0/1067803059 wait complete. 2024-02-21T23:25:50.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:50.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.284+0000 7f1e258de640 1 Processor -- start 2024-02-21T23:25:50.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.284+0000 7f1e258de640 1 -- start start 2024-02-21T23:25:50.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.285+0000 7f1e258de640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1e20154600 0x7f1e200cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:50.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.285+0000 7f1e258de640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2015ab40 con 0x7f1e200c6f50 2024-02-21T23:25:50.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.285+0000 7f1e258de640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2015ae00 con 0x7f1e200c7c10 2024-02-21T23:25:50.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.285+0000 7f1e258de640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e200c74a0 con 0x7f1e20154600 2024-02-21T23:25:50.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.285+0000 7f1e1effd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1e20154600 0x7f1e200cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:50.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.285+0000 7f1e1e7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1e200c6f50 0x7f1e200ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60454/0 (socket says 172.21.15.133:60454) 2024-02-21T23:25:50.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.285+0000 7f1e1e7fc640 1 -- 172.21.15.133:0/3799354764 learned_addr learned my addr 172.21.15.133:0/3799354764 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:50.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.288+0000 7f1e1dffb640 1 -- 172.21.15.133:0/3799354764 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567447430 0 0) 0x7f1e2015ab40 con 0x7f1e200c6f50 2024-02-21T23:25:50.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.288+0000 7f1e1dffb640 1 -- 172.21.15.133:0/3799354764 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e08003520 con 0x7f1e200c6f50 2024-02-21T23:25:50.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.291+0000 7f1e1effd640 1 -- 172.21.15.133:0/3799354764 >> v1:172.21.15.142:6789/0 conn(0x7f1e200c7c10 legacy=0x7f1e200c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.291+0000 7f1e1effd640 1 -- 172.21.15.133:0/3799354764 >> v1:172.21.15.133:6789/0 conn(0x7f1e200c6f50 legacy=0x7f1e200ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.291+0000 7f1e1effd640 1 -- 172.21.15.133:0/3799354764 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200cb580 con 0x7f1e20154600 2024-02-21T23:25:50.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.295+0000 7f1e1effd640 1 --2- 172.21.15.133:0/3799354764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e20154600 0x7f1e200cbc00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e100097a0 tx=0x7f1e1002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ccd3292d43236b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:50.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.299+0000 7f1e1dffb640 1 -- 172.21.15.133:0/3799354764 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10034070 con 0x7f1e20154600 2024-02-21T23:25:50.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.299+0000 7f1e1dffb640 1 -- 172.21.15.133:0/3799354764 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e10030610 con 0x7f1e20154600 2024-02-21T23:25:50.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.299+0000 7f1e1dffb640 1 -- 172.21.15.133:0/3799354764 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e100309e0 con 0x7f1e20154600 2024-02-21T23:25:50.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.300+0000 7f1e258de640 1 -- 172.21.15.133:0/3799354764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e20154600 msgr2=0x7f1e200cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.300+0000 7f1e258de640 1 --2- 172.21.15.133:0/3799354764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e20154600 0x7f1e200cbc00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e100097a0 tx=0x7f1e1002b960 comp rx=0 tx=0).stop 2024-02-21T23:25:50.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.300+0000 7f1e258de640 1 -- 172.21.15.133:0/3799354764 shutdown_connections 2024-02-21T23:25:50.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.300+0000 7f1e258de640 1 --2- 172.21.15.133:0/3799354764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e20154600 0x7f1e200cbc00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.300+0000 7f1e258de640 1 -- 172.21.15.133:0/3799354764 >> 172.21.15.133:0/3799354764 conn(0x7f1e20062000 msgr2=0x7f1e200cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:50.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.300+0000 7f1e258de640 1 -- 172.21.15.133:0/3799354764 shutdown_connections 2024-02-21T23:25:50.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.300+0000 7f1e258de640 1 -- 172.21.15.133:0/3799354764 wait complete. 2024-02-21T23:25:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.301+0000 7f1e258de640 1 Processor -- start 2024-02-21T23:25:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.301+0000 7f1e258de640 1 -- start start 2024-02-21T23:25:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.302+0000 7f1e258de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200c6f50 0x7f1e20153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.302+0000 7f1e258de640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e200c7c10 0x7f1e201e3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.302+0000 7f1e258de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e20154600 0x7f1e2006b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.302+0000 7f1e258de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200b5720 con 0x7f1e200c6f50 2024-02-21T23:25:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.302+0000 7f1e258de640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200c7780 con 0x7f1e200c7c10 2024-02-21T23:25:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.302+0000 7f1e258de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200b58a0 con 0x7f1e20154600 2024-02-21T23:25:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.302+0000 7f1e1effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200c6f50 0x7f1e20153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:50.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.302+0000 7f1e1e7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e200c7c10 0x7f1e201e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:50.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.302+0000 7f1e1f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e20154600 0x7f1e2006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:50.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.303+0000 7f1e1effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200c6f50 0x7f1e20153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56002/0 (socket says 172.21.15.133:56002) 2024-02-21T23:25:50.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.303+0000 7f1e1effd640 1 -- 172.21.15.133:0/2797969900 learned_addr learned my addr 172.21.15.133:0/2797969900 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:50.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.306+0000 7f1e1effd640 1 -- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e20154600 msgr2=0x7f1e2006b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.306+0000 7f1e1effd640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e20154600 0x7f1e2006b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.306+0000 7f1e1effd640 1 -- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e200c7c10 msgr2=0x7f1e201e3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.306+0000 7f1e1effd640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e200c7c10 0x7f1e201e3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.306+0000 7f1e1effd640 1 -- 172.21.15.133:0/2797969900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2015a1a0 con 0x7f1e200c6f50 2024-02-21T23:25:50.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.308+0000 7f1e1effd640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200c6f50 0x7f1e20153a40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e100098c0 tx=0x7f1e100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:50.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.312+0000 7f1e07fff640 1 -- 172.21.15.133:0/2797969900 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10034040 con 0x7f1e200c6f50 2024-02-21T23:25:50.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.312+0000 7f1e07fff640 1 -- 172.21.15.133:0/2797969900 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e10039990 con 0x7f1e200c6f50 2024-02-21T23:25:50.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.312+0000 7f1e07fff640 1 -- 172.21.15.133:0/2797969900 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10039da0 con 0x7f1e200c6f50 2024-02-21T23:25:50.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.312+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e20157440 con 0x7f1e200c6f50 2024-02-21T23:25:50.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.312+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e201e5570 con 0x7f1e200c6f50 2024-02-21T23:25:50.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.313+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2006a130 con 0x7f1e200c6f50 2024-02-21T23:25:50.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.560+0000 7f1e07fff640 1 -- 172.21.15.133:0/2797969900 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e10037070 con 0x7f1e200c6f50 2024-02-21T23:25:50.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.561+0000 7f1e07fff640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:50.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.561+0000 7f1e1e7fc640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:50.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.562+0000 7f1e1e7fc640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e201526b0 tx=0x7f1e14009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.575+0000 7f1e07fff640 1 -- 172.21.15.133:0/2797969900 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1e100792a0 con 0x7f1e200c6f50 2024-02-21T23:25:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.575+0000 7f1e07fff640 1 -- 172.21.15.133:0/2797969900 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f1e1007b200 con 0x7f1e200c6f50 2024-02-21T23:25:50.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.762+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e20158be0 con 0x7f1e200c6f50 2024-02-21T23:25:50.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.808+0000 7f1e07fff640 1 -- 172.21.15.133:0/2797969900 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f1e1007f2a0 con 0x7f1e200c6f50 2024-02-21T23:25:50.795 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:50.795 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:50.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e0c06b500 msgr2=0x7f1e0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e201526b0 tx=0x7f1e14009040 comp rx=0 tx=0).stop 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200c6f50 msgr2=0x7f1e20153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200c6f50 0x7f1e20153a40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e100098c0 tx=0x7f1e100046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 shutdown_connections 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e20154600 0x7f1e2006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e200c7c10 0x7f1e201e3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 --2- 172.21.15.133:0/2797969900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200c6f50 0x7f1e20153a40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.810+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 >> 172.21.15.133:0/2797969900 conn(0x7f1e20062000 msgr2=0x7f1e2014dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.811+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 shutdown_connections 2024-02-21T23:25:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.811+0000 7f1e258de640 1 -- 172.21.15.133:0/2797969900 wait complete. 2024-02-21T23:25:50.809 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-21T23:25:50.809 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-21T23:25:50.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.929+0000 7f3ad535e640 1 Processor -- start 2024-02-21T23:25:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.929+0000 7f3ad535e640 1 -- start start 2024-02-21T23:25:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.930+0000 7f3ad535e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3ad015e140 0x7f3ad0165940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.930+0000 7f3ad535e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad00c9d20 con 0x7f3ad015aa30 2024-02-21T23:25:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.930+0000 7f3ad535e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad00c96c0 con 0x7f3ad01540d0 2024-02-21T23:25:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.930+0000 7f3ad535e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad015d5a0 con 0x7f3ad015e140 2024-02-21T23:25:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.931+0000 7f3ace7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3ad015aa30 0x7f3ad015da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60470/0 (socket says 172.21.15.133:60470) 2024-02-21T23:25:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.931+0000 7f3ace7fc640 1 -- 172.21.15.133:0/2771517945 learned_addr learned my addr 172.21.15.133:0/2771517945 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.931+0000 7f3acf7fe640 1 --2- 172.21.15.133:0/2771517945 >> v2:172.21.15.142:3300/0 conn(0x7f3ad015e140 0x7f3ad0165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.934+0000 7f3acdffb640 1 -- 172.21.15.133:0/2771517945 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576055610 0 0) 0x7f3ad00c9d20 con 0x7f3ad015aa30 2024-02-21T23:25:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.934+0000 7f3acdffb640 1 -- 172.21.15.133:0/2771517945 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab8003520 con 0x7f3ad015aa30 2024-02-21T23:25:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.935+0000 7f3acf7fe640 1 -- 172.21.15.133:0/2771517945 >> v1:172.21.15.142:6789/0 conn(0x7f3ad01540d0 legacy=0x7f3ad00cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.935+0000 7f3acf7fe640 1 -- 172.21.15.133:0/2771517945 >> v1:172.21.15.133:6789/0 conn(0x7f3ad015aa30 legacy=0x7f3ad015da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.935+0000 7f3acf7fe640 1 -- 172.21.15.133:0/2771517945 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad00c9fc0 con 0x7f3ad015e140 2024-02-21T23:25:50.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.940+0000 7f3acf7fe640 1 --2- 172.21.15.133:0/2771517945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad015e140 0x7f3ad0165940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009d90 tx=0x7f3ac802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afe5b40a638a2c27 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:50.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.944+0000 7f3acdffb640 1 -- 172.21.15.133:0/2771517945 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8039070 con 0x7f3ad015e140 2024-02-21T23:25:50.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.944+0000 7f3acdffb640 1 -- 172.21.15.133:0/2771517945 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac802cc60 con 0x7f3ad015e140 2024-02-21T23:25:50.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.944+0000 7f3acdffb640 1 -- 172.21.15.133:0/2771517945 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8034660 con 0x7f3ad015e140 2024-02-21T23:25:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.945+0000 7f3ad535e640 1 -- 172.21.15.133:0/2771517945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad015e140 msgr2=0x7f3ad0165940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.945+0000 7f3ad535e640 1 --2- 172.21.15.133:0/2771517945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad015e140 0x7f3ad0165940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009d90 tx=0x7f3ac802beb0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.945+0000 7f3ad535e640 1 -- 172.21.15.133:0/2771517945 shutdown_connections 2024-02-21T23:25:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.945+0000 7f3ad535e640 1 --2- 172.21.15.133:0/2771517945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad015e140 0x7f3ad0165940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.945+0000 7f3ad535e640 1 -- 172.21.15.133:0/2771517945 >> 172.21.15.133:0/2771517945 conn(0x7f3ad006f7f0 msgr2=0x7f3ad0061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.945+0000 7f3ad535e640 1 -- 172.21.15.133:0/2771517945 shutdown_connections 2024-02-21T23:25:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.945+0000 7f3ad535e640 1 -- 172.21.15.133:0/2771517945 wait complete. 2024-02-21T23:25:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.946+0000 7f3ad535e640 1 Processor -- start 2024-02-21T23:25:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.947+0000 7f3ad535e640 1 -- start start 2024-02-21T23:25:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.947+0000 7f3ad535e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad01540d0 0x7f3ad00cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.947+0000 7f3ad535e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ad015aa30 0x7f3ad0159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.947+0000 7f3ad535e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad015e140 0x7f3ad0147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.947+0000 7f3ad535e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00ce010 con 0x7f3ad015e140 2024-02-21T23:25:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.947+0000 7f3ad535e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad006c040 con 0x7f3ad01540d0 2024-02-21T23:25:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.947+0000 7f3ad535e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00ce190 con 0x7f3ad015aa30 2024-02-21T23:25:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.948+0000 7f3aceffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad01540d0 0x7f3ad00cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.948+0000 7f3ace7fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ad015aa30 0x7f3ad0159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.948+0000 7f3aceffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad01540d0 0x7f3ad00cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:56282/0 (socket says 172.21.15.133:56282) 2024-02-21T23:25:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.948+0000 7f3aceffd640 1 -- 172.21.15.133:0/1147240346 learned_addr learned my addr 172.21.15.133:0/1147240346 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.949+0000 7f3acf7fe640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad015e140 0x7f3ad0147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.952+0000 7f3ace7fc640 1 -- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad01540d0 msgr2=0x7f3ad00cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.952+0000 7f3ace7fc640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad01540d0 0x7f3ad00cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.952+0000 7f3ace7fc640 1 -- 172.21.15.133:0/1147240346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad015e140 msgr2=0x7f3ad0147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.952+0000 7f3ace7fc640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad015e140 0x7f3ad0147370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.952+0000 7f3ace7fc640 1 -- 172.21.15.133:0/1147240346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0157fc0 con 0x7f3ad015aa30 2024-02-21T23:25:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.954+0000 7f3ace7fc640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ad015aa30 0x7f3ad0159060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c430 tx=0x7f3ac400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.958+0000 7f3ab77fe640 1 -- 172.21.15.133:0/1147240346 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4017070 con 0x7f3ad015aa30 2024-02-21T23:25:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.958+0000 7f3ab77fe640 1 -- 172.21.15.133:0/1147240346 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4005ce0 con 0x7f3ad015aa30 2024-02-21T23:25:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.958+0000 7f3ab77fe640 1 -- 172.21.15.133:0/1147240346 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40072f0 con 0x7f3ad015aa30 2024-02-21T23:25:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.958+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad01582a0 con 0x7f3ad015aa30 2024-02-21T23:25:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.958+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad0158860 con 0x7f3ad015aa30 2024-02-21T23:25:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:50.960+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad006a130 con 0x7f3ad015aa30 2024-02-21T23:25:51.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.214+0000 7f3ab77fe640 1 -- 172.21.15.133:0/1147240346 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ac401a070 con 0x7f3ad015aa30 2024-02-21T23:25:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.214+0000 7f3ab77fe640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3abc06b500 0x7f3abc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.214+0000 7f3aceffd640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3abc06b500 0x7f3abc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.215+0000 7f3aceffd640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3abc06b500 0x7f3abc06d9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0001840 tx=0x7f3ac0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.219+0000 7f3ab77fe640 1 -- 172.21.15.133:0/1147240346 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ac40561f0 con 0x7f3ad015aa30 2024-02-21T23:25:51.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.230+0000 7f3ab77fe640 1 -- 172.21.15.133:0/1147240346 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f3ac4021020 con 0x7f3ad015aa30 2024-02-21T23:25:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.421+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ad00c8f20 con 0x7f3abc06b500 2024-02-21T23:25:51.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.430+0000 7f3ab77fe640 1 -- 172.21.15.133:0/1147240346 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332612 (secure 0 0 0) 0x7f3ad00c8f20 con 0x7f3abc06b500 2024-02-21T23:25:51.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:51.423 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-02-21T23:25:51.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3abc06b500 msgr2=0x7f3abc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3abc06b500 0x7f3abc06d9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0001840 tx=0x7f3ac0009040 comp rx=0 tx=0).stop 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ad015aa30 msgr2=0x7f3ad0159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ad015aa30 0x7f3ad0159060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c430 tx=0x7f3ac400c8f0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 shutdown_connections 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3abc06b500 0x7f3abc06d9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad015e140 0x7f3ad0147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ad015aa30 0x7f3ad0159060 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 --2- 172.21.15.133:0/1147240346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad01540d0 0x7f3ad00cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 >> 172.21.15.133:0/1147240346 conn(0x7f3ad006f7f0 msgr2=0x7f3ad0152300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 shutdown_connections 2024-02-21T23:25:51.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.433+0000 7f3ad535e640 1 -- 172.21.15.133:0/1147240346 wait complete. 2024-02-21T23:25:51.431 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-02-21T23:25:50.691540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218332,"kb_used_data":2176,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756388,"statfs":{"total":773094113280,"available":772870541312,"internally_reserved":0,"allocated":2228224,"data_stored":1992064,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002481"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637827+0000","last_change":"2024-02-21T23:25:16.134390+0000","last_active":"2024-02-21T23:25:22.637827+0000","last_peered":"2024-02-21T23:25:22.637827+0000","last_clean":"2024-02-21T23:25:22.637827+0000","last_became_active":"2024-02-21T23:25:16.134112+0000","last_became_peered":"2024-02-21T23:25:16.134112+0000","last_unstale":"2024-02-21T23:25:22.637827+0000","last_undegraded":"2024-02-21T23:25:22.637827+0000","last_fullsized":"2024-02-21T23:25:22.637827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195309+0000","last_change":"2024-02-21T23:25:17.135738+0000","last_active":"2024-02-21T23:25:23.195309+0000","last_peered":"2024-02-21T23:25:23.195309+0000","last_clean":"2024-02-21T23:25:23.195309+0000","last_became_active":"2024-02-21T23:25:17.135586+0000","last_became_peered":"2024-02-21T23:25:17.135586+0000","last_unstale":"2024-02-21T23:25:23.195309+0000","last_undegraded":"2024-02-21T23:25:23.195309+0000","last_fullsized":"2024-02-21T23:25:23.195309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194873+0000","last_change":"2024-02-21T23:25:22.248962+0000","last_active":"2024-02-21T23:25:23.194873+0000","last_peered":"2024-02-21T23:25:23.194873+0000","last_clean":"2024-02-21T23:25:23.194873+0000","last_became_active":"2024-02-21T23:25:22.248610+0000","last_became_peered":"2024-02-21T23:25:22.248610+0000","last_unstale":"2024-02-21T23:25:23.194873+0000","last_undegraded":"2024-02-21T23:25:23.194873+0000","last_fullsized":"2024-02-21T23:25:23.194873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:45.939036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195369+0000","last_change":"2024-02-21T23:25:16.123635+0000","last_active":"2024-02-21T23:25:23.195369+0000","last_peered":"2024-02-21T23:25:23.195369+0000","last_clean":"2024-02-21T23:25:23.195369+0000","last_became_active":"2024-02-21T23:25:16.123374+0000","last_became_peered":"2024-02-21T23:25:16.123374+0000","last_unstale":"2024-02-21T23:25:23.195369+0000","last_undegraded":"2024-02-21T23:25:23.195369+0000","last_fullsized":"2024-02-21T23:25:23.195369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637647+0000","last_change":"2024-02-21T23:25:17.192858+0000","last_active":"2024-02-21T23:25:22.637647+0000","last_peered":"2024-02-21T23:25:22.637647+0000","last_clean":"2024-02-21T23:25:22.637647+0000","last_became_active":"2024-02-21T23:25:17.192626+0000","last_became_peered":"2024-02-21T23:25:17.192626+0000","last_unstale":"2024-02-21T23:25:22.637647+0000","last_undegraded":"2024-02-21T23:25:22.637647+0000","last_fullsized":"2024-02-21T23:25:22.637647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195812+0000","last_change":"2024-02-21T23:25:21.683490+0000","last_active":"2024-02-21T23:25:23.195812+0000","last_peered":"2024-02-21T23:25:23.195812+0000","last_clean":"2024-02-21T23:25:23.195812+0000","last_became_active":"2024-02-21T23:25:21.683173+0000","last_became_peered":"2024-02-21T23:25:21.683173+0000","last_unstale":"2024-02-21T23:25:23.195812+0000","last_undegraded":"2024-02-21T23:25:23.195812+0000","last_fullsized":"2024-02-21T23:25:23.195812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:49:01.803167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196175+0000","last_change":"2024-02-21T23:25:16.134805+0000","last_active":"2024-02-21T23:25:23.196175+0000","last_peered":"2024-02-21T23:25:23.196175+0000","last_clean":"2024-02-21T23:25:23.196175+0000","last_became_active":"2024-02-21T23:25:16.134533+0000","last_became_peered":"2024-02-21T23:25:16.134533+0000","last_unstale":"2024-02-21T23:25:23.196175+0000","last_undegraded":"2024-02-21T23:25:23.196175+0000","last_fullsized":"2024-02-21T23:25:23.196175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197229+0000","last_change":"2024-02-21T23:25:17.182131+0000","last_active":"2024-02-21T23:25:23.197229+0000","last_peered":"2024-02-21T23:25:23.197229+0000","last_clean":"2024-02-21T23:25:23.197229+0000","last_became_active":"2024-02-21T23:25:17.181886+0000","last_became_peered":"2024-02-21T23:25:17.181886+0000","last_unstale":"2024-02-21T23:25:23.197229+0000","last_undegraded":"2024-02-21T23:25:23.197229+0000","last_fullsized":"2024-02-21T23:25:23.197229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195843+0000","last_change":"2024-02-21T23:25:21.683506+0000","last_active":"2024-02-21T23:25:23.195843+0000","last_peered":"2024-02-21T23:25:23.195843+0000","last_clean":"2024-02-21T23:25:23.195843+0000","last_became_active":"2024-02-21T23:25:21.683115+0000","last_became_peered":"2024-02-21T23:25:21.683115+0000","last_unstale":"2024-02-21T23:25:23.195843+0000","last_undegraded":"2024-02-21T23:25:23.195843+0000","last_fullsized":"2024-02-21T23:25:23.195843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:13.865145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195380+0000","last_change":"2024-02-21T23:25:16.130851+0000","last_active":"2024-02-21T23:25:23.195380+0000","last_peered":"2024-02-21T23:25:23.195380+0000","last_clean":"2024-02-21T23:25:23.195380+0000","last_became_active":"2024-02-21T23:25:16.130604+0000","last_became_peered":"2024-02-21T23:25:16.130604+0000","last_unstale":"2024-02-21T23:25:23.195380+0000","last_undegraded":"2024-02-21T23:25:23.195380+0000","last_fullsized":"2024-02-21T23:25:23.195380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197689+0000","last_change":"2024-02-21T23:25:17.192642+0000","last_active":"2024-02-21T23:25:23.197689+0000","last_peered":"2024-02-21T23:25:23.197689+0000","last_clean":"2024-02-21T23:25:23.197689+0000","last_became_active":"2024-02-21T23:25:17.192394+0000","last_became_peered":"2024-02-21T23:25:17.192394+0000","last_unstale":"2024-02-21T23:25:23.197689+0000","last_undegraded":"2024-02-21T23:25:23.197689+0000","last_fullsized":"2024-02-21T23:25:23.197689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195564+0000","last_change":"2024-02-21T23:25:21.679491+0000","last_active":"2024-02-21T23:25:23.195564+0000","last_peered":"2024-02-21T23:25:23.195564+0000","last_clean":"2024-02-21T23:25:23.195564+0000","last_became_active":"2024-02-21T23:25:21.679117+0000","last_became_peered":"2024-02-21T23:25:21.679117+0000","last_unstale":"2024-02-21T23:25:23.195564+0000","last_undegraded":"2024-02-21T23:25:23.195564+0000","last_fullsized":"2024-02-21T23:25:23.195564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:52.680971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670455+0000","last_change":"2024-02-21T23:25:17.161438+0000","last_active":"2024-02-21T23:25:22.670455+0000","last_peered":"2024-02-21T23:25:22.670455+0000","last_clean":"2024-02-21T23:25:22.670455+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:25:22.670455+0000","last_undegraded":"2024-02-21T23:25:22.670455+0000","last_fullsized":"2024-02-21T23:25:22.670455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196550+0000","last_change":"2024-02-21T23:25:16.136913+0000","last_active":"2024-02-21T23:25:23.196550+0000","last_peered":"2024-02-21T23:25:23.196550+0000","last_clean":"2024-02-21T23:25:23.196550+0000","last_became_active":"2024-02-21T23:25:16.136628+0000","last_became_peered":"2024-02-21T23:25:16.136628+0000","last_unstale":"2024-02-21T23:25:23.196550+0000","last_undegraded":"2024-02-21T23:25:23.196550+0000","last_fullsized":"2024-02-21T23:25:23.196550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196588+0000","last_change":"2024-02-21T23:25:21.683706+0000","last_active":"2024-02-21T23:25:23.196588+0000","last_peered":"2024-02-21T23:25:23.196588+0000","last_clean":"2024-02-21T23:25:23.196588+0000","last_became_active":"2024-02-21T23:25:21.683116+0000","last_became_peered":"2024-02-21T23:25:21.683116+0000","last_unstale":"2024-02-21T23:25:23.196588+0000","last_undegraded":"2024-02-21T23:25:23.196588+0000","last_fullsized":"2024-02-21T23:25:23.196588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:06.265383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195016+0000","last_change":"2024-02-21T23:25:17.137829+0000","last_active":"2024-02-21T23:25:23.195016+0000","last_peered":"2024-02-21T23:25:23.195016+0000","last_clean":"2024-02-21T23:25:23.195016+0000","last_became_active":"2024-02-21T23:25:17.137557+0000","last_became_peered":"2024-02-21T23:25:17.137557+0000","last_unstale":"2024-02-21T23:25:23.195016+0000","last_undegraded":"2024-02-21T23:25:23.195016+0000","last_fullsized":"2024-02-21T23:25:23.195016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723448+0000","last_change":"2024-02-21T23:25:16.146181+0000","last_active":"2024-02-21T23:25:22.723448+0000","last_peered":"2024-02-21T23:25:22.723448+0000","last_clean":"2024-02-21T23:25:22.723448+0000","last_became_active":"2024-02-21T23:25:16.145932+0000","last_became_peered":"2024-02-21T23:25:16.145932+0000","last_unstale":"2024-02-21T23:25:22.723448+0000","last_undegraded":"2024-02-21T23:25:22.723448+0000","last_fullsized":"2024-02-21T23:25:22.723448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196544+0000","last_change":"2024-02-21T23:25:21.686044+0000","last_active":"2024-02-21T23:25:23.196544+0000","last_peered":"2024-02-21T23:25:23.196544+0000","last_clean":"2024-02-21T23:25:23.196544+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:25:23.196544+0000","last_undegraded":"2024-02-21T23:25:23.196544+0000","last_fullsized":"2024-02-21T23:25:23.196544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:39.740454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195078+0000","last_change":"2024-02-21T23:25:17.190236+0000","last_active":"2024-02-21T23:25:23.195078+0000","last_peered":"2024-02-21T23:25:23.195078+0000","last_clean":"2024-02-21T23:25:23.195078+0000","last_became_active":"2024-02-21T23:25:17.189957+0000","last_became_peered":"2024-02-21T23:25:17.189957+0000","last_unstale":"2024-02-21T23:25:23.195078+0000","last_undegraded":"2024-02-21T23:25:23.195078+0000","last_fullsized":"2024-02-21T23:25:23.195078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723142+0000","last_change":"2024-02-21T23:25:16.148108+0000","last_active":"2024-02-21T23:25:22.723142+0000","last_peered":"2024-02-21T23:25:22.723142+0000","last_clean":"2024-02-21T23:25:22.723142+0000","last_became_active":"2024-02-21T23:25:16.147930+0000","last_became_peered":"2024-02-21T23:25:16.147930+0000","last_unstale":"2024-02-21T23:25:22.723142+0000","last_undegraded":"2024-02-21T23:25:22.723142+0000","last_fullsized":"2024-02-21T23:25:22.723142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723970+0000","last_change":"2024-02-21T23:25:22.248150+0000","last_active":"2024-02-21T23:25:22.723970+0000","last_peered":"2024-02-21T23:25:22.723970+0000","last_clean":"2024-02-21T23:25:22.723970+0000","last_became_active":"2024-02-21T23:25:22.247769+0000","last_became_peered":"2024-02-21T23:25:22.247769+0000","last_unstale":"2024-02-21T23:25:22.723970+0000","last_undegraded":"2024-02-21T23:25:22.723970+0000","last_fullsized":"2024-02-21T23:25:22.723970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196252+0000","last_change":"2024-02-21T23:25:17.189758+0000","last_active":"2024-02-21T23:25:23.196252+0000","last_peered":"2024-02-21T23:25:23.196252+0000","last_clean":"2024-02-21T23:25:23.196252+0000","last_became_active":"2024-02-21T23:25:17.189492+0000","last_became_peered":"2024-02-21T23:25:17.189492+0000","last_unstale":"2024-02-21T23:25:23.196252+0000","last_undegraded":"2024-02-21T23:25:23.196252+0000","last_fullsized":"2024-02-21T23:25:23.196252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:42.526598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195319+0000","last_change":"2024-02-21T23:25:16.131611+0000","last_active":"2024-02-21T23:25:23.195319+0000","last_peered":"2024-02-21T23:25:23.195319+0000","last_clean":"2024-02-21T23:25:23.195319+0000","last_became_active":"2024-02-21T23:25:16.131158+0000","last_became_peered":"2024-02-21T23:25:16.131158+0000","last_unstale":"2024-02-21T23:25:23.195319+0000","last_undegraded":"2024-02-21T23:25:23.195319+0000","last_fullsized":"2024-02-21T23:25:23.195319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196480+0000","last_change":"2024-02-21T23:25:21.683821+0000","last_active":"2024-02-21T23:25:23.196480+0000","last_peered":"2024-02-21T23:25:23.196480+0000","last_clean":"2024-02-21T23:25:23.196480+0000","last_became_active":"2024-02-21T23:25:21.683466+0000","last_became_peered":"2024-02-21T23:25:21.683466+0000","last_unstale":"2024-02-21T23:25:23.196480+0000","last_undegraded":"2024-02-21T23:25:23.196480+0000","last_fullsized":"2024-02-21T23:25:23.196480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637653+0000","last_change":"2024-02-21T23:25:17.191394+0000","last_active":"2024-02-21T23:25:22.637653+0000","last_peered":"2024-02-21T23:25:22.637653+0000","last_clean":"2024-02-21T23:25:22.637653+0000","last_became_active":"2024-02-21T23:25:17.191106+0000","last_became_peered":"2024-02-21T23:25:17.191106+0000","last_unstale":"2024-02-21T23:25:22.637653+0000","last_undegraded":"2024-02-21T23:25:22.637653+0000","last_fullsized":"2024-02-21T23:25:22.637653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670984+0000","last_change":"2024-02-21T23:25:16.134862+0000","last_active":"2024-02-21T23:25:22.670984+0000","last_peered":"2024-02-21T23:25:22.670984+0000","last_clean":"2024-02-21T23:25:22.670984+0000","last_became_active":"2024-02-21T23:25:16.134602+0000","last_became_peered":"2024-02-21T23:25:16.134602+0000","last_unstale":"2024-02-21T23:25:22.670984+0000","last_undegraded":"2024-02-21T23:25:22.670984+0000","last_fullsized":"2024-02-21T23:25:22.670984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196150+0000","last_change":"2024-02-21T23:25:22.249606+0000","last_active":"2024-02-21T23:25:23.196150+0000","last_peered":"2024-02-21T23:25:23.196150+0000","last_clean":"2024-02-21T23:25:23.196150+0000","last_became_active":"2024-02-21T23:25:22.249214+0000","last_became_peered":"2024-02-21T23:25:22.249214+0000","last_unstale":"2024-02-21T23:25:23.196150+0000","last_undegraded":"2024-02-21T23:25:23.196150+0000","last_fullsized":"2024-02-21T23:25:23.196150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:36.929076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195603+0000","last_change":"2024-02-21T23:25:17.183290+0000","last_active":"2024-02-21T23:25:23.195603+0000","last_peered":"2024-02-21T23:25:23.195603+0000","last_clean":"2024-02-21T23:25:23.195603+0000","last_became_active":"2024-02-21T23:25:17.183047+0000","last_became_peered":"2024-02-21T23:25:17.183047+0000","last_unstale":"2024-02-21T23:25:23.195603+0000","last_undegraded":"2024-02-21T23:25:23.195603+0000","last_fullsized":"2024-02-21T23:25:23.195603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723977+0000","last_change":"2024-02-21T23:25:16.146202+0000","last_active":"2024-02-21T23:25:22.723977+0000","last_peered":"2024-02-21T23:25:22.723977+0000","last_clean":"2024-02-21T23:25:22.723977+0000","last_became_active":"2024-02-21T23:25:16.145911+0000","last_became_peered":"2024-02-21T23:25:16.145911+0000","last_unstale":"2024-02-21T23:25:22.723977+0000","last_undegraded":"2024-02-21T23:25:22.723977+0000","last_fullsized":"2024-02-21T23:25:22.723977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194768+0000","last_change":"2024-02-21T23:25:22.243297+0000","last_active":"2024-02-21T23:25:23.194768+0000","last_peered":"2024-02-21T23:25:23.194768+0000","last_clean":"2024-02-21T23:25:23.194768+0000","last_became_active":"2024-02-21T23:25:22.242922+0000","last_became_peered":"2024-02-21T23:25:22.242922+0000","last_unstale":"2024-02-21T23:25:23.194768+0000","last_undegraded":"2024-02-21T23:25:23.194768+0000","last_fullsized":"2024-02-21T23:25:23.194768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:52:51.773574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723650+0000","last_change":"2024-02-21T23:25:17.190577+0000","last_active":"2024-02-21T23:25:22.723650+0000","last_peered":"2024-02-21T23:25:22.723650+0000","last_clean":"2024-02-21T23:25:22.723650+0000","last_became_active":"2024-02-21T23:25:17.190332+0000","last_became_peered":"2024-02-21T23:25:17.190332+0000","last_unstale":"2024-02-21T23:25:22.723650+0000","last_undegraded":"2024-02-21T23:25:22.723650+0000","last_fullsized":"2024-02-21T23:25:22.723650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196263+0000","last_change":"2024-02-21T23:25:16.134076+0000","last_active":"2024-02-21T23:25:23.196263+0000","last_peered":"2024-02-21T23:25:23.196263+0000","last_clean":"2024-02-21T23:25:23.196263+0000","last_became_active":"2024-02-21T23:25:16.133816+0000","last_became_peered":"2024-02-21T23:25:16.133816+0000","last_unstale":"2024-02-21T23:25:23.196263+0000","last_undegraded":"2024-02-21T23:25:23.196263+0000","last_fullsized":"2024-02-21T23:25:23.196263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637332+0000","last_change":"2024-02-21T23:25:22.243360+0000","last_active":"2024-02-21T23:25:22.637332+0000","last_peered":"2024-02-21T23:25:22.637332+0000","last_clean":"2024-02-21T23:25:22.637332+0000","last_became_active":"2024-02-21T23:25:22.242958+0000","last_became_peered":"2024-02-21T23:25:22.242958+0000","last_unstale":"2024-02-21T23:25:22.637332+0000","last_undegraded":"2024-02-21T23:25:22.637332+0000","last_fullsized":"2024-02-21T23:25:22.637332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197769+0000","last_change":"2024-02-21T23:25:17.192402+0000","last_active":"2024-02-21T23:25:23.197769+0000","last_peered":"2024-02-21T23:25:23.197769+0000","last_clean":"2024-02-21T23:25:23.197769+0000","last_became_active":"2024-02-21T23:25:17.192138+0000","last_became_peered":"2024-02-21T23:25:17.192138+0000","last_unstale":"2024-02-21T23:25:23.197769+0000","last_undegraded":"2024-02-21T23:25:23.197769+0000","last_fullsized":"2024-02-21T23:25:23.197769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723671+0000","last_change":"2024-02-21T23:25:16.145521+0000","last_active":"2024-02-21T23:25:22.723671+0000","last_peered":"2024-02-21T23:25:22.723671+0000","last_clean":"2024-02-21T23:25:22.723671+0000","last_became_active":"2024-02-21T23:25:16.145255+0000","last_became_peered":"2024-02-21T23:25:16.145255+0000","last_unstale":"2024-02-21T23:25:22.723671+0000","last_undegraded":"2024-02-21T23:25:22.723671+0000","last_fullsized":"2024-02-21T23:25:22.723671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723238+0000","last_change":"2024-02-21T23:25:21.683849+0000","last_active":"2024-02-21T23:25:22.723238+0000","last_peered":"2024-02-21T23:25:22.723238+0000","last_clean":"2024-02-21T23:25:22.723238+0000","last_became_active":"2024-02-21T23:25:21.683379+0000","last_became_peered":"2024-02-21T23:25:21.683379+0000","last_unstale":"2024-02-21T23:25:22.723238+0000","last_undegraded":"2024-02-21T23:25:22.723238+0000","last_fullsized":"2024-02-21T23:25:22.723238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196877+0000","last_change":"2024-02-21T23:25:17.166232+0000","last_active":"2024-02-21T23:25:23.196877+0000","last_peered":"2024-02-21T23:25:23.196877+0000","last_clean":"2024-02-21T23:25:23.196877+0000","last_became_active":"2024-02-21T23:25:17.165999+0000","last_became_peered":"2024-02-21T23:25:17.165999+0000","last_unstale":"2024-02-21T23:25:23.196877+0000","last_undegraded":"2024-02-21T23:25:23.196877+0000","last_fullsized":"2024-02-21T23:25:23.196877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:45.125986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197593+0000","last_change":"2024-02-21T23:25:16.147082+0000","last_active":"2024-02-21T23:25:23.197593+0000","last_peered":"2024-02-21T23:25:23.197593+0000","last_clean":"2024-02-21T23:25:23.197593+0000","last_became_active":"2024-02-21T23:25:16.146797+0000","last_became_peered":"2024-02-21T23:25:16.146797+0000","last_unstale":"2024-02-21T23:25:23.197593+0000","last_undegraded":"2024-02-21T23:25:23.197593+0000","last_fullsized":"2024-02-21T23:25:23.197593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723612+0000","last_change":"2024-02-21T23:25:21.682860+0000","last_active":"2024-02-21T23:25:22.723612+0000","last_peered":"2024-02-21T23:25:22.723612+0000","last_clean":"2024-02-21T23:25:22.723612+0000","last_became_active":"2024-02-21T23:25:21.682404+0000","last_became_peered":"2024-02-21T23:25:21.682404+0000","last_unstale":"2024-02-21T23:25:22.723612+0000","last_undegraded":"2024-02-21T23:25:22.723612+0000","last_fullsized":"2024-02-21T23:25:22.723612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723262+0000","last_change":"2024-02-21T23:25:17.190425+0000","last_active":"2024-02-21T23:25:22.723262+0000","last_peered":"2024-02-21T23:25:22.723262+0000","last_clean":"2024-02-21T23:25:22.723262+0000","last_became_active":"2024-02-21T23:25:17.190172+0000","last_became_peered":"2024-02-21T23:25:17.190172+0000","last_unstale":"2024-02-21T23:25:22.723262+0000","last_undegraded":"2024-02-21T23:25:22.723262+0000","last_fullsized":"2024-02-21T23:25:22.723262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636897+0000","last_change":"2024-02-21T23:25:16.108263+0000","last_active":"2024-02-21T23:25:22.636897+0000","last_peered":"2024-02-21T23:25:22.636897+0000","last_clean":"2024-02-21T23:25:22.636897+0000","last_became_active":"2024-02-21T23:25:16.107822+0000","last_became_peered":"2024-02-21T23:25:16.107822+0000","last_unstale":"2024-02-21T23:25:22.636897+0000","last_undegraded":"2024-02-21T23:25:22.636897+0000","last_fullsized":"2024-02-21T23:25:22.636897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723957+0000","last_change":"2024-02-21T23:25:21.686034+0000","last_active":"2024-02-21T23:25:22.723957+0000","last_peered":"2024-02-21T23:25:22.723957+0000","last_clean":"2024-02-21T23:25:22.723957+0000","last_became_active":"2024-02-21T23:25:21.685546+0000","last_became_peered":"2024-02-21T23:25:21.685546+0000","last_unstale":"2024-02-21T23:25:22.723957+0000","last_undegraded":"2024-02-21T23:25:22.723957+0000","last_fullsized":"2024-02-21T23:25:22.723957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636614+0000","last_change":"2024-02-21T23:25:17.161738+0000","last_active":"2024-02-21T23:25:22.636614+0000","last_peered":"2024-02-21T23:25:22.636614+0000","last_clean":"2024-02-21T23:25:22.636614+0000","last_became_active":"2024-02-21T23:25:17.161459+0000","last_became_peered":"2024-02-21T23:25:17.161459+0000","last_unstale":"2024-02-21T23:25:22.636614+0000","last_undegraded":"2024-02-21T23:25:22.636614+0000","last_fullsized":"2024-02-21T23:25:22.636614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:31:58.434561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195408+0000","last_change":"2024-02-21T23:25:16.147162+0000","last_active":"2024-02-21T23:25:23.195408+0000","last_peered":"2024-02-21T23:25:23.195408+0000","last_clean":"2024-02-21T23:25:23.195408+0000","last_became_active":"2024-02-21T23:25:16.146850+0000","last_became_peered":"2024-02-21T23:25:16.146850+0000","last_unstale":"2024-02-21T23:25:23.195408+0000","last_undegraded":"2024-02-21T23:25:23.195408+0000","last_fullsized":"2024-02-21T23:25:23.195408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637976+0000","last_change":"2024-02-21T23:25:22.248109+0000","last_active":"2024-02-21T23:25:22.637976+0000","last_peered":"2024-02-21T23:25:22.637976+0000","last_clean":"2024-02-21T23:25:22.637976+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:25:22.637976+0000","last_undegraded":"2024-02-21T23:25:22.637976+0000","last_fullsized":"2024-02-21T23:25:22.637976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194934+0000","last_change":"2024-02-21T23:25:17.192450+0000","last_active":"2024-02-21T23:25:23.194934+0000","last_peered":"2024-02-21T23:25:23.194934+0000","last_clean":"2024-02-21T23:25:23.194934+0000","last_became_active":"2024-02-21T23:25:17.192167+0000","last_became_peered":"2024-02-21T23:25:17.192167+0000","last_unstale":"2024-02-21T23:25:23.194934+0000","last_undegraded":"2024-02-21T23:25:23.194934+0000","last_fullsized":"2024-02-21T23:25:23.194934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197488+0000","last_change":"2024-02-21T23:25:16.127688+0000","last_active":"2024-02-21T23:25:23.197488+0000","last_peered":"2024-02-21T23:25:23.197488+0000","last_clean":"2024-02-21T23:25:23.197488+0000","last_became_active":"2024-02-21T23:25:16.127388+0000","last_became_peered":"2024-02-21T23:25:16.127388+0000","last_unstale":"2024-02-21T23:25:23.197488+0000","last_undegraded":"2024-02-21T23:25:23.197488+0000","last_fullsized":"2024-02-21T23:25:23.197488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195483+0000","last_change":"2024-02-21T23:25:05.612817+0000","last_active":"2024-02-21T23:25:23.195483+0000","last_peered":"2024-02-21T23:25:23.195483+0000","last_clean":"2024-02-21T23:25:23.195483+0000","last_became_active":"2024-02-21T23:25:05.612483+0000","last_became_peered":"2024-02-21T23:25:05.612483+0000","last_unstale":"2024-02-21T23:25:23.195483+0000","last_undegraded":"2024-02-21T23:25:23.195483+0000","last_fullsized":"2024-02-21T23:25:23.195483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_clean_scrub_stamp":"2024-02-21T23:25:03.736732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:23:33.073040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670664+0000","last_change":"2024-02-21T23:25:22.248174+0000","last_active":"2024-02-21T23:25:22.670664+0000","last_peered":"2024-02-21T23:25:22.670664+0000","last_clean":"2024-02-21T23:25:22.670664+0000","last_became_active":"2024-02-21T23:25:22.247834+0000","last_became_peered":"2024-02-21T23:25:22.247834+0000","last_unstale":"2024-02-21T23:25:22.670664+0000","last_undegraded":"2024-02-21T23:25:22.670664+0000","last_fullsized":"2024-02-21T23:25:22.670664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:13.953358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196092+0000","last_change":"2024-02-21T23:25:17.192575+0000","last_active":"2024-02-21T23:25:23.196092+0000","last_peered":"2024-02-21T23:25:23.196092+0000","last_clean":"2024-02-21T23:25:23.196092+0000","last_became_active":"2024-02-21T23:25:17.192293+0000","last_became_peered":"2024-02-21T23:25:17.192293+0000","last_unstale":"2024-02-21T23:25:23.196092+0000","last_undegraded":"2024-02-21T23:25:23.196092+0000","last_fullsized":"2024-02-21T23:25:23.196092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637073+0000","last_change":"2024-02-21T23:25:16.128369+0000","last_active":"2024-02-21T23:25:22.637073+0000","last_peered":"2024-02-21T23:25:22.637073+0000","last_clean":"2024-02-21T23:25:22.637073+0000","last_became_active":"2024-02-21T23:25:16.128067+0000","last_became_peered":"2024-02-21T23:25:16.128067+0000","last_unstale":"2024-02-21T23:25:22.637073+0000","last_undegraded":"2024-02-21T23:25:22.637073+0000","last_fullsized":"2024-02-21T23:25:22.637073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670470+0000","last_change":"2024-02-21T23:25:22.243282+0000","last_active":"2024-02-21T23:25:22.670470+0000","last_peered":"2024-02-21T23:25:22.670470+0000","last_clean":"2024-02-21T23:25:22.670470+0000","last_became_active":"2024-02-21T23:25:22.242916+0000","last_became_peered":"2024-02-21T23:25:22.242916+0000","last_unstale":"2024-02-21T23:25:22.670470+0000","last_undegraded":"2024-02-21T23:25:22.670470+0000","last_fullsized":"2024-02-21T23:25:22.670470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197322+0000","last_change":"2024-02-21T23:25:17.185418+0000","last_active":"2024-02-21T23:25:23.197322+0000","last_peered":"2024-02-21T23:25:23.197322+0000","last_clean":"2024-02-21T23:25:23.197322+0000","last_became_active":"2024-02-21T23:25:17.185119+0000","last_became_peered":"2024-02-21T23:25:17.185119+0000","last_unstale":"2024-02-21T23:25:23.197322+0000","last_undegraded":"2024-02-21T23:25:23.197322+0000","last_fullsized":"2024-02-21T23:25:23.197322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723756+0000","last_change":"2024-02-21T23:25:16.151088+0000","last_active":"2024-02-21T23:25:22.723756+0000","last_peered":"2024-02-21T23:25:22.723756+0000","last_clean":"2024-02-21T23:25:22.723756+0000","last_became_active":"2024-02-21T23:25:16.150819+0000","last_became_peered":"2024-02-21T23:25:16.150819+0000","last_unstale":"2024-02-21T23:25:22.723756+0000","last_undegraded":"2024-02-21T23:25:22.723756+0000","last_fullsized":"2024-02-21T23:25:22.723756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637420+0000","last_change":"2024-02-21T23:25:21.684592+0000","last_active":"2024-02-21T23:25:22.637420+0000","last_peered":"2024-02-21T23:25:22.637420+0000","last_clean":"2024-02-21T23:25:22.637420+0000","last_became_active":"2024-02-21T23:25:21.684192+0000","last_became_peered":"2024-02-21T23:25:21.684192+0000","last_unstale":"2024-02-21T23:25:22.637420+0000","last_undegraded":"2024-02-21T23:25:22.637420+0000","last_fullsized":"2024-02-21T23:25:22.637420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196248+0000","last_change":"2024-02-21T23:25:17.184268+0000","last_active":"2024-02-21T23:25:23.196248+0000","last_peered":"2024-02-21T23:25:23.196248+0000","last_clean":"2024-02-21T23:25:23.196248+0000","last_became_active":"2024-02-21T23:25:17.184027+0000","last_became_peered":"2024-02-21T23:25:17.184027+0000","last_unstale":"2024-02-21T23:25:23.196248+0000","last_undegraded":"2024-02-21T23:25:23.196248+0000","last_fullsized":"2024-02-21T23:25:23.196248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637140+0000","last_change":"2024-02-21T23:25:16.108099+0000","last_active":"2024-02-21T23:25:22.637140+0000","last_peered":"2024-02-21T23:25:22.637140+0000","last_clean":"2024-02-21T23:25:22.637140+0000","last_became_active":"2024-02-21T23:25:16.107815+0000","last_became_peered":"2024-02-21T23:25:16.107815+0000","last_unstale":"2024-02-21T23:25:22.637140+0000","last_undegraded":"2024-02-21T23:25:22.637140+0000","last_fullsized":"2024-02-21T23:25:22.637140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195214+0000","last_change":"2024-02-21T23:25:22.248926+0000","last_active":"2024-02-21T23:25:23.195214+0000","last_peered":"2024-02-21T23:25:23.195214+0000","last_clean":"2024-02-21T23:25:23.195214+0000","last_became_active":"2024-02-21T23:25:22.248558+0000","last_became_peered":"2024-02-21T23:25:22.248558+0000","last_unstale":"2024-02-21T23:25:23.195214+0000","last_undegraded":"2024-02-21T23:25:23.195214+0000","last_fullsized":"2024-02-21T23:25:23.195214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195709+0000","last_change":"2024-02-21T23:25:17.183254+0000","last_active":"2024-02-21T23:25:23.195709+0000","last_peered":"2024-02-21T23:25:23.195709+0000","last_clean":"2024-02-21T23:25:23.195709+0000","last_became_active":"2024-02-21T23:25:17.183020+0000","last_became_peered":"2024-02-21T23:25:17.183020+0000","last_unstale":"2024-02-21T23:25:23.195709+0000","last_undegraded":"2024-02-21T23:25:23.195709+0000","last_fullsized":"2024-02-21T23:25:23.195709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196256+0000","last_change":"2024-02-21T23:25:16.129131+0000","last_active":"2024-02-21T23:25:23.196256+0000","last_peered":"2024-02-21T23:25:23.196256+0000","last_clean":"2024-02-21T23:25:23.196256+0000","last_became_active":"2024-02-21T23:25:16.128853+0000","last_became_peered":"2024-02-21T23:25:16.128853+0000","last_unstale":"2024-02-21T23:25:23.196256+0000","last_undegraded":"2024-02-21T23:25:23.196256+0000","last_fullsized":"2024-02-21T23:25:23.196256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:29.302181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195886+0000","last_change":"2024-02-21T23:25:21.684342+0000","last_active":"2024-02-21T23:25:23.195886+0000","last_peered":"2024-02-21T23:25:23.195886+0000","last_clean":"2024-02-21T23:25:23.195886+0000","last_became_active":"2024-02-21T23:25:21.683991+0000","last_became_peered":"2024-02-21T23:25:21.683991+0000","last_unstale":"2024-02-21T23:25:23.195886+0000","last_undegraded":"2024-02-21T23:25:23.195886+0000","last_fullsized":"2024-02-21T23:25:23.195886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:21:00.312314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195905+0000","last_change":"2024-02-21T23:25:17.183442+0000","last_active":"2024-02-21T23:25:23.195905+0000","last_peered":"2024-02-21T23:25:23.195905+0000","last_clean":"2024-02-21T23:25:23.195905+0000","last_became_active":"2024-02-21T23:25:17.183057+0000","last_became_peered":"2024-02-21T23:25:17.183057+0000","last_unstale":"2024-02-21T23:25:23.195905+0000","last_undegraded":"2024-02-21T23:25:23.195905+0000","last_fullsized":"2024-02-21T23:25:23.195905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636480+0000","last_change":"2024-02-21T23:25:16.145801+0000","last_active":"2024-02-21T23:25:22.636480+0000","last_peered":"2024-02-21T23:25:22.636480+0000","last_clean":"2024-02-21T23:25:22.636480+0000","last_became_active":"2024-02-21T23:25:16.145523+0000","last_became_peered":"2024-02-21T23:25:16.145523+0000","last_unstale":"2024-02-21T23:25:22.636480+0000","last_undegraded":"2024-02-21T23:25:22.636480+0000","last_fullsized":"2024-02-21T23:25:22.636480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637169+0000","last_change":"2024-02-21T23:25:21.679191+0000","last_active":"2024-02-21T23:25:22.637169+0000","last_peered":"2024-02-21T23:25:22.637169+0000","last_clean":"2024-02-21T23:25:22.637169+0000","last_became_active":"2024-02-21T23:25:21.678715+0000","last_became_peered":"2024-02-21T23:25:21.678715+0000","last_unstale":"2024-02-21T23:25:22.637169+0000","last_undegraded":"2024-02-21T23:25:22.637169+0000","last_fullsized":"2024-02-21T23:25:22.637169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670230+0000","last_change":"2024-02-21T23:25:17.138401+0000","last_active":"2024-02-21T23:25:22.670230+0000","last_peered":"2024-02-21T23:25:22.670230+0000","last_clean":"2024-02-21T23:25:22.670230+0000","last_became_active":"2024-02-21T23:25:17.138136+0000","last_became_peered":"2024-02-21T23:25:17.138136+0000","last_unstale":"2024-02-21T23:25:22.670230+0000","last_undegraded":"2024-02-21T23:25:22.670230+0000","last_fullsized":"2024-02-21T23:25:22.670230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670228+0000","last_change":"2024-02-21T23:25:16.130209+0000","last_active":"2024-02-21T23:25:22.670228+0000","last_peered":"2024-02-21T23:25:22.670228+0000","last_clean":"2024-02-21T23:25:22.670228+0000","last_became_active":"2024-02-21T23:25:16.129974+0000","last_became_peered":"2024-02-21T23:25:16.129974+0000","last_unstale":"2024-02-21T23:25:22.670228+0000","last_undegraded":"2024-02-21T23:25:22.670228+0000","last_fullsized":"2024-02-21T23:25:22.670228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195559+0000","last_change":"2024-02-21T23:25:22.243303+0000","last_active":"2024-02-21T23:25:23.195559+0000","last_peered":"2024-02-21T23:25:23.195559+0000","last_clean":"2024-02-21T23:25:23.195559+0000","last_became_active":"2024-02-21T23:25:22.242960+0000","last_became_peered":"2024-02-21T23:25:22.242960+0000","last_unstale":"2024-02-21T23:25:23.195559+0000","last_undegraded":"2024-02-21T23:25:23.195559+0000","last_fullsized":"2024-02-21T23:25:23.195559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:54:16.386354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196403+0000","last_change":"2024-02-21T23:25:17.184233+0000","last_active":"2024-02-21T23:25:23.196403+0000","last_peered":"2024-02-21T23:25:23.196403+0000","last_clean":"2024-02-21T23:25:23.196403+0000","last_became_active":"2024-02-21T23:25:17.183980+0000","last_became_peered":"2024-02-21T23:25:17.183980+0000","last_unstale":"2024-02-21T23:25:23.196403+0000","last_undegraded":"2024-02-21T23:25:23.196403+0000","last_fullsized":"2024-02-21T23:25:23.196403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197566+0000","last_change":"2024-02-21T23:25:16.131147+0000","last_active":"2024-02-21T23:25:23.197566+0000","last_peered":"2024-02-21T23:25:23.197566+0000","last_clean":"2024-02-21T23:25:23.197566+0000","last_became_active":"2024-02-21T23:25:16.130553+0000","last_became_peered":"2024-02-21T23:25:16.130553+0000","last_unstale":"2024-02-21T23:25:23.197566+0000","last_undegraded":"2024-02-21T23:25:23.197566+0000","last_fullsized":"2024-02-21T23:25:23.197566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195579+0000","last_change":"2024-02-21T23:25:22.243261+0000","last_active":"2024-02-21T23:25:23.195579+0000","last_peered":"2024-02-21T23:25:23.195579+0000","last_clean":"2024-02-21T23:25:23.195579+0000","last_became_active":"2024-02-21T23:25:22.242900+0000","last_became_peered":"2024-02-21T23:25:22.242900+0000","last_unstale":"2024-02-21T23:25:23.195579+0000","last_undegraded":"2024-02-21T23:25:23.195579+0000","last_fullsized":"2024-02-21T23:25:23.195579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:34:44.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637437+0000","last_change":"2024-02-21T23:25:17.183303+0000","last_active":"2024-02-21T23:25:22.637437+0000","last_peered":"2024-02-21T23:25:22.637437+0000","last_clean":"2024-02-21T23:25:22.637437+0000","last_became_active":"2024-02-21T23:25:17.183042+0000","last_became_peered":"2024-02-21T23:25:17.183042+0000","last_unstale":"2024-02-21T23:25:22.637437+0000","last_undegraded":"2024-02-21T23:25:22.637437+0000","last_fullsized":"2024-02-21T23:25:22.637437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637466+0000","last_change":"2024-02-21T23:25:16.127609+0000","last_active":"2024-02-21T23:25:22.637466+0000","last_peered":"2024-02-21T23:25:22.637466+0000","last_clean":"2024-02-21T23:25:22.637466+0000","last_became_active":"2024-02-21T23:25:16.127375+0000","last_became_peered":"2024-02-21T23:25:16.127375+0000","last_unstale":"2024-02-21T23:25:22.637466+0000","last_undegraded":"2024-02-21T23:25:22.637466+0000","last_fullsized":"2024-02-21T23:25:22.637466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197762+0000","last_change":"2024-02-21T23:25:22.248329+0000","last_active":"2024-02-21T23:25:23.197762+0000","last_peered":"2024-02-21T23:25:23.197762+0000","last_clean":"2024-02-21T23:25:23.197762+0000","last_became_active":"2024-02-21T23:25:22.247889+0000","last_became_peered":"2024-02-21T23:25:22.247889+0000","last_unstale":"2024-02-21T23:25:23.197762+0000","last_undegraded":"2024-02-21T23:25:23.197762+0000","last_fullsized":"2024-02-21T23:25:23.197762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196057+0000","last_change":"2024-02-21T23:25:17.183741+0000","last_active":"2024-02-21T23:25:23.196057+0000","last_peered":"2024-02-21T23:25:23.196057+0000","last_clean":"2024-02-21T23:25:23.196057+0000","last_became_active":"2024-02-21T23:25:17.183408+0000","last_became_peered":"2024-02-21T23:25:17.183408+0000","last_unstale":"2024-02-21T23:25:23.196057+0000","last_undegraded":"2024-02-21T23:25:23.196057+0000","last_fullsized":"2024-02-21T23:25:23.196057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:38.383407+0000","last_change":"2024-02-21T23:25:16.116460+0000","last_active":"2024-02-21T23:25:38.383407+0000","last_peered":"2024-02-21T23:25:38.383407+0000","last_clean":"2024-02-21T23:25:38.383407+0000","last_became_active":"2024-02-21T23:25:16.116162+0000","last_became_peered":"2024-02-21T23:25:16.116162+0000","last_unstale":"2024-02-21T23:25:38.383407+0000","last_undegraded":"2024-02-21T23:25:38.383407+0000","last_fullsized":"2024-02-21T23:25:38.383407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723404+0000","last_change":"2024-02-21T23:25:21.683199+0000","last_active":"2024-02-21T23:25:22.723404+0000","last_peered":"2024-02-21T23:25:22.723404+0000","last_clean":"2024-02-21T23:25:22.723404+0000","last_became_active":"2024-02-21T23:25:21.682425+0000","last_became_peered":"2024-02-21T23:25:21.682425+0000","last_unstale":"2024-02-21T23:25:22.723404+0000","last_undegraded":"2024-02-21T23:25:22.723404+0000","last_fullsized":"2024-02-21T23:25:22.723404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196728+0000","last_change":"2024-02-21T23:25:17.184837+0000","last_active":"2024-02-21T23:25:23.196728+0000","last_peered":"2024-02-21T23:25:23.196728+0000","last_clean":"2024-02-21T23:25:23.196728+0000","last_became_active":"2024-02-21T23:25:17.184574+0000","last_became_peered":"2024-02-21T23:25:17.184574+0000","last_unstale":"2024-02-21T23:25:23.196728+0000","last_undegraded":"2024-02-21T23:25:23.196728+0000","last_fullsized":"2024-02-21T23:25:23.196728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723604+0000","last_change":"2024-02-21T23:25:16.151031+0000","last_active":"2024-02-21T23:25:22.723604+0000","last_peered":"2024-02-21T23:25:22.723604+0000","last_clean":"2024-02-21T23:25:22.723604+0000","last_became_active":"2024-02-21T23:25:16.150767+0000","last_became_peered":"2024-02-21T23:25:16.150767+0000","last_unstale":"2024-02-21T23:25:22.723604+0000","last_undegraded":"2024-02-21T23:25:22.723604+0000","last_fullsized":"2024-02-21T23:25:22.723604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196772+0000","last_change":"2024-02-21T23:25:21.683995+0000","last_active":"2024-02-21T23:25:23.196772+0000","last_peered":"2024-02-21T23:25:23.196772+0000","last_clean":"2024-02-21T23:25:23.196772+0000","last_became_active":"2024-02-21T23:25:21.683415+0000","last_became_peered":"2024-02-21T23:25:21.683415+0000","last_unstale":"2024-02-21T23:25:23.196772+0000","last_undegraded":"2024-02-21T23:25:23.196772+0000","last_fullsized":"2024-02-21T23:25:23.196772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637736+0000","last_change":"2024-02-21T23:25:17.187230+0000","last_active":"2024-02-21T23:25:22.637736+0000","last_peered":"2024-02-21T23:25:22.637736+0000","last_clean":"2024-02-21T23:25:22.637736+0000","last_became_active":"2024-02-21T23:25:17.186952+0000","last_became_peered":"2024-02-21T23:25:17.186952+0000","last_unstale":"2024-02-21T23:25:22.637736+0000","last_undegraded":"2024-02-21T23:25:22.637736+0000","last_fullsized":"2024-02-21T23:25:22.637736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197393+0000","last_change":"2024-02-21T23:25:16.134722+0000","last_active":"2024-02-21T23:25:23.197393+0000","last_peered":"2024-02-21T23:25:23.197393+0000","last_clean":"2024-02-21T23:25:23.197393+0000","last_became_active":"2024-02-21T23:25:16.134458+0000","last_became_peered":"2024-02-21T23:25:16.134458+0000","last_unstale":"2024-02-21T23:25:23.197393+0000","last_undegraded":"2024-02-21T23:25:23.197393+0000","last_fullsized":"2024-02-21T23:25:23.197393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195006+0000","last_change":"2024-02-21T23:25:21.684109+0000","last_active":"2024-02-21T23:25:23.195006+0000","last_peered":"2024-02-21T23:25:23.195006+0000","last_clean":"2024-02-21T23:25:23.195006+0000","last_became_active":"2024-02-21T23:25:21.683705+0000","last_became_peered":"2024-02-21T23:25:21.683705+0000","last_unstale":"2024-02-21T23:25:23.195006+0000","last_undegraded":"2024-02-21T23:25:23.195006+0000","last_fullsized":"2024-02-21T23:25:23.195006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197237+0000","last_change":"2024-02-21T23:25:17.188374+0000","last_active":"2024-02-21T23:25:23.197237+0000","last_peered":"2024-02-21T23:25:23.197237+0000","last_clean":"2024-02-21T23:25:23.197237+0000","last_became_active":"2024-02-21T23:25:17.187792+0000","last_became_peered":"2024-02-21T23:25:17.187792+0000","last_unstale":"2024-02-21T23:25:23.197237+0000","last_undegraded":"2024-02-21T23:25:23.197237+0000","last_fullsized":"2024-02-21T23:25:23.197237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670904+0000","last_change":"2024-02-21T23:25:16.129446+0000","last_active":"2024-02-21T23:25:22.670904+0000","last_peered":"2024-02-21T23:25:22.670904+0000","last_clean":"2024-02-21T23:25:22.670904+0000","last_became_active":"2024-02-21T23:25:16.129164+0000","last_became_peered":"2024-02-21T23:25:16.129164+0000","last_unstale":"2024-02-21T23:25:22.670904+0000","last_undegraded":"2024-02-21T23:25:22.670904+0000","last_fullsized":"2024-02-21T23:25:22.670904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194890+0000","last_change":"2024-02-21T23:25:22.248096+0000","last_active":"2024-02-21T23:25:23.194890+0000","last_peered":"2024-02-21T23:25:23.194890+0000","last_clean":"2024-02-21T23:25:23.194890+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:25:23.194890+0000","last_undegraded":"2024-02-21T23:25:23.194890+0000","last_fullsized":"2024-02-21T23:25:23.194890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637006+0000","last_change":"2024-02-21T23:25:22.248119+0000","last_active":"2024-02-21T23:25:22.637006+0000","last_peered":"2024-02-21T23:25:22.637006+0000","last_clean":"2024-02-21T23:25:22.637006+0000","last_became_active":"2024-02-21T23:25:22.247811+0000","last_became_peered":"2024-02-21T23:25:22.247811+0000","last_unstale":"2024-02-21T23:25:22.637006+0000","last_undegraded":"2024-02-21T23:25:22.637006+0000","last_fullsized":"2024-02-21T23:25:22.637006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723036+0000","last_change":"2024-02-21T23:25:17.189176+0000","last_active":"2024-02-21T23:25:22.723036+0000","last_peered":"2024-02-21T23:25:22.723036+0000","last_clean":"2024-02-21T23:25:22.723036+0000","last_became_active":"2024-02-21T23:25:17.188885+0000","last_became_peered":"2024-02-21T23:25:17.188885+0000","last_unstale":"2024-02-21T23:25:22.723036+0000","last_undegraded":"2024-02-21T23:25:22.723036+0000","last_fullsized":"2024-02-21T23:25:22.723036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195502+0000","last_change":"2024-02-21T23:25:16.132036+0000","last_active":"2024-02-21T23:25:23.195502+0000","last_peered":"2024-02-21T23:25:23.195502+0000","last_clean":"2024-02-21T23:25:23.195502+0000","last_became_active":"2024-02-21T23:25:16.131756+0000","last_became_peered":"2024-02-21T23:25:16.131756+0000","last_unstale":"2024-02-21T23:25:23.195502+0000","last_undegraded":"2024-02-21T23:25:23.195502+0000","last_fullsized":"2024-02-21T23:25:23.195502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670902+0000","last_change":"2024-02-21T23:25:21.686133+0000","last_active":"2024-02-21T23:25:22.670902+0000","last_peered":"2024-02-21T23:25:22.670902+0000","last_clean":"2024-02-21T23:25:22.670902+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:25:22.670902+0000","last_undegraded":"2024-02-21T23:25:22.670902+0000","last_fullsized":"2024-02-21T23:25:22.670902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194875+0000","last_change":"2024-02-21T23:25:17.135158+0000","last_active":"2024-02-21T23:25:23.194875+0000","last_peered":"2024-02-21T23:25:23.194875+0000","last_clean":"2024-02-21T23:25:23.194875+0000","last_became_active":"2024-02-21T23:25:17.134888+0000","last_became_peered":"2024-02-21T23:25:17.134888+0000","last_unstale":"2024-02-21T23:25:23.194875+0000","last_undegraded":"2024-02-21T23:25:23.194875+0000","last_fullsized":"2024-02-21T23:25:23.194875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194948+0000","last_change":"2024-02-21T23:25:16.116378+0000","last_active":"2024-02-21T23:25:23.194948+0000","last_peered":"2024-02-21T23:25:23.194948+0000","last_clean":"2024-02-21T23:25:23.194948+0000","last_became_active":"2024-02-21T23:25:16.116201+0000","last_became_peered":"2024-02-21T23:25:16.116201+0000","last_unstale":"2024-02-21T23:25:23.194948+0000","last_undegraded":"2024-02-21T23:25:23.194948+0000","last_fullsized":"2024-02-21T23:25:23.194948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670541+0000","last_change":"2024-02-21T23:25:16.132314+0000","last_active":"2024-02-21T23:25:22.670541+0000","last_peered":"2024-02-21T23:25:22.670541+0000","last_clean":"2024-02-21T23:25:22.670541+0000","last_became_active":"2024-02-21T23:25:16.132062+0000","last_became_peered":"2024-02-21T23:25:16.132062+0000","last_unstale":"2024-02-21T23:25:22.670541+0000","last_undegraded":"2024-02-21T23:25:22.670541+0000","last_fullsized":"2024-02-21T23:25:22.670541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637458+0000","last_change":"2024-02-21T23:25:17.161451+0000","last_active":"2024-02-21T23:25:22.637458+0000","last_peered":"2024-02-21T23:25:22.637458+0000","last_clean":"2024-02-21T23:25:22.637458+0000","last_became_active":"2024-02-21T23:25:17.161176+0000","last_became_peered":"2024-02-21T23:25:17.161176+0000","last_unstale":"2024-02-21T23:25:22.637458+0000","last_undegraded":"2024-02-21T23:25:22.637458+0000","last_fullsized":"2024-02-21T23:25:22.637458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636872+0000","last_change":"2024-02-21T23:25:21.685724+0000","last_active":"2024-02-21T23:25:22.636872+0000","last_peered":"2024-02-21T23:25:22.636872+0000","last_clean":"2024-02-21T23:25:22.636872+0000","last_became_active":"2024-02-21T23:25:21.685375+0000","last_became_peered":"2024-02-21T23:25:21.685375+0000","last_unstale":"2024-02-21T23:25:22.636872+0000","last_undegraded":"2024-02-21T23:25:22.636872+0000","last_fullsized":"2024-02-21T23:25:22.636872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:56:28.795401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194995+0000","last_change":"2024-02-21T23:25:16.134127+0000","last_active":"2024-02-21T23:25:23.194995+0000","last_peered":"2024-02-21T23:25:23.194995+0000","last_clean":"2024-02-21T23:25:23.194995+0000","last_became_active":"2024-02-21T23:25:16.133839+0000","last_became_peered":"2024-02-21T23:25:16.133839+0000","last_unstale":"2024-02-21T23:25:23.194995+0000","last_undegraded":"2024-02-21T23:25:23.194995+0000","last_fullsized":"2024-02-21T23:25:23.194995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195317+0000","last_change":"2024-02-21T23:25:17.190455+0000","last_active":"2024-02-21T23:25:23.195317+0000","last_peered":"2024-02-21T23:25:23.195317+0000","last_clean":"2024-02-21T23:25:23.195317+0000","last_became_active":"2024-02-21T23:25:17.190194+0000","last_became_peered":"2024-02-21T23:25:17.190194+0000","last_unstale":"2024-02-21T23:25:23.195317+0000","last_undegraded":"2024-02-21T23:25:23.195317+0000","last_fullsized":"2024-02-21T23:25:23.195317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:03.451409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195261+0000","last_change":"2024-02-21T23:25:21.683501+0000","last_active":"2024-02-21T23:25:23.195261+0000","last_peered":"2024-02-21T23:25:23.195261+0000","last_clean":"2024-02-21T23:25:23.195261+0000","last_became_active":"2024-02-21T23:25:21.683139+0000","last_became_peered":"2024-02-21T23:25:21.683139+0000","last_unstale":"2024-02-21T23:25:23.195261+0000","last_undegraded":"2024-02-21T23:25:23.195261+0000","last_fullsized":"2024-02-21T23:25:23.195261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-22T23:58:15.072875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670359+0000","last_change":"2024-02-21T23:25:17.138039+0000","last_active":"2024-02-21T23:25:22.670359+0000","last_peered":"2024-02-21T23:25:22.670359+0000","last_clean":"2024-02-21T23:25:22.670359+0000","last_became_active":"2024-02-21T23:25:17.137717+0000","last_became_peered":"2024-02-21T23:25:17.137717+0000","last_unstale":"2024-02-21T23:25:22.670359+0000","last_undegraded":"2024-02-21T23:25:22.670359+0000","last_fullsized":"2024-02-21T23:25:22.670359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636758+0000","last_change":"2024-02-21T23:25:17.194727+0000","last_active":"2024-02-21T23:25:22.636758+0000","last_peered":"2024-02-21T23:25:22.636758+0000","last_clean":"2024-02-21T23:25:22.636758+0000","last_became_active":"2024-02-21T23:25:17.194533+0000","last_became_peered":"2024-02-21T23:25:17.194533+0000","last_unstale":"2024-02-21T23:25:22.636758+0000","last_undegraded":"2024-02-21T23:25:22.636758+0000","last_fullsized":"2024-02-21T23:25:22.636758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197507+0000","last_change":"2024-02-21T23:25:17.191051+0000","last_active":"2024-02-21T23:25:23.197507+0000","last_peered":"2024-02-21T23:25:23.197507+0000","last_clean":"2024-02-21T23:25:23.197507+0000","last_became_active":"2024-02-21T23:25:17.190600+0000","last_became_peered":"2024-02-21T23:25:17.190600+0000","last_unstale":"2024-02-21T23:25:23.197507+0000","last_undegraded":"2024-02-21T23:25:23.197507+0000","last_fullsized":"2024-02-21T23:25:23.197507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637377+0000","last_change":"2024-02-21T23:25:17.135311+0000","last_active":"2024-02-21T23:25:22.637377+0000","last_peered":"2024-02-21T23:25:22.637377+0000","last_clean":"2024-02-21T23:25:22.637377+0000","last_became_active":"2024-02-21T23:25:17.135011+0000","last_became_peered":"2024-02-21T23:25:17.135011+0000","last_unstale":"2024-02-21T23:25:22.637377+0000","last_undegraded":"2024-02-21T23:25:22.637377+0000","last_fullsized":"2024-02-21T23:25:22.637377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195670+0000","last_change":"2024-02-21T23:25:17.161696+0000","last_active":"2024-02-21T23:25:23.195670+0000","last_peered":"2024-02-21T23:25:23.195670+0000","last_clean":"2024-02-21T23:25:23.195670+0000","last_became_active":"2024-02-21T23:25:17.161401+0000","last_became_peered":"2024-02-21T23:25:17.161401+0000","last_unstale":"2024-02-21T23:25:23.195670+0000","last_undegraded":"2024-02-21T23:25:23.195670+0000","last_fullsized":"2024-02-21T23:25:23.195670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723507+0000","last_change":"2024-02-21T23:25:17.189731+0000","last_active":"2024-02-21T23:25:22.723507+0000","last_peered":"2024-02-21T23:25:22.723507+0000","last_clean":"2024-02-21T23:25:22.723507+0000","last_became_active":"2024-02-21T23:25:17.189484+0000","last_became_peered":"2024-02-21T23:25:17.189484+0000","last_unstale":"2024-02-21T23:25:22.723507+0000","last_undegraded":"2024-02-21T23:25:22.723507+0000","last_fullsized":"2024-02-21T23:25:22.723507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670963+0000","last_change":"2024-02-21T23:25:17.184125+0000","last_active":"2024-02-21T23:25:22.670963+0000","last_peered":"2024-02-21T23:25:22.670963+0000","last_clean":"2024-02-21T23:25:22.670963+0000","last_became_active":"2024-02-21T23:25:17.183865+0000","last_became_peered":"2024-02-21T23:25:17.183865+0000","last_unstale":"2024-02-21T23:25:22.670963+0000","last_undegraded":"2024-02-21T23:25:22.670963+0000","last_fullsized":"2024-02-21T23:25:22.670963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195515+0000","last_change":"2024-02-21T23:25:17.161482+0000","last_active":"2024-02-21T23:25:23.195515+0000","last_peered":"2024-02-21T23:25:23.195515+0000","last_clean":"2024-02-21T23:25:23.195515+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:25:23.195515+0000","last_undegraded":"2024-02-21T23:25:23.195515+0000","last_fullsized":"2024-02-21T23:25:23.195515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195414+0000","last_change":"2024-02-21T23:25:17.185691+0000","last_active":"2024-02-21T23:25:23.195414+0000","last_peered":"2024-02-21T23:25:23.195414+0000","last_clean":"2024-02-21T23:25:23.195414+0000","last_became_active":"2024-02-21T23:25:17.185420+0000","last_became_peered":"2024-02-21T23:25:17.185420+0000","last_unstale":"2024-02-21T23:25:23.195414+0000","last_undegraded":"2024-02-21T23:25:23.195414+0000","last_fullsized":"2024-02-21T23:25:23.195414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194975+0000","last_change":"2024-02-21T23:25:17.161321+0000","last_active":"2024-02-21T23:25:23.194975+0000","last_peered":"2024-02-21T23:25:23.194975+0000","last_clean":"2024-02-21T23:25:23.194975+0000","last_became_active":"2024-02-21T23:25:17.161010+0000","last_became_peered":"2024-02-21T23:25:17.161010+0000","last_unstale":"2024-02-21T23:25:23.194975+0000","last_undegraded":"2024-02-21T23:25:23.194975+0000","last_fullsized":"2024-02-21T23:25:23.194975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197503+0000","last_change":"2024-02-21T23:25:17.191072+0000","last_active":"2024-02-21T23:25:23.197503+0000","last_peered":"2024-02-21T23:25:23.197503+0000","last_clean":"2024-02-21T23:25:23.197503+0000","last_became_active":"2024-02-21T23:25:17.187741+0000","last_became_peered":"2024-02-21T23:25:17.187741+0000","last_unstale":"2024-02-21T23:25:23.197503+0000","last_undegraded":"2024-02-21T23:25:23.197503+0000","last_fullsized":"2024-02-21T23:25:23.197503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637229+0000","last_change":"2024-02-21T23:25:17.186616+0000","last_active":"2024-02-21T23:25:22.637229+0000","last_peered":"2024-02-21T23:25:22.637229+0000","last_clean":"2024-02-21T23:25:22.637229+0000","last_became_active":"2024-02-21T23:25:17.186108+0000","last_became_peered":"2024-02-21T23:25:17.186108+0000","last_unstale":"2024-02-21T23:25:22.637229+0000","last_undegraded":"2024-02-21T23:25:22.637229+0000","last_fullsized":"2024-02-21T23:25:22.637229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670481+0000","last_change":"2024-02-21T23:25:17.161428+0000","last_active":"2024-02-21T23:25:22.670481+0000","last_peered":"2024-02-21T23:25:22.670481+0000","last_clean":"2024-02-21T23:25:22.670481+0000","last_became_active":"2024-02-21T23:25:17.161140+0000","last_became_peered":"2024-02-21T23:25:17.161140+0000","last_unstale":"2024-02-21T23:25:22.670481+0000","last_undegraded":"2024-02-21T23:25:22.670481+0000","last_fullsized":"2024-02-21T23:25:22.670481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196229+0000","last_change":"2024-02-21T23:25:17.190666+0000","last_active":"2024-02-21T23:25:23.196229+0000","last_peered":"2024-02-21T23:25:23.196229+0000","last_clean":"2024-02-21T23:25:23.196229+0000","last_became_active":"2024-02-21T23:25:17.190420+0000","last_became_peered":"2024-02-21T23:25:17.190420+0000","last_unstale":"2024-02-21T23:25:23.196229+0000","last_undegraded":"2024-02-21T23:25:23.196229+0000","last_fullsized":"2024-02-21T23:25:23.196229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637103+0000","last_change":"2024-02-21T23:25:17.189583+0000","last_active":"2024-02-21T23:25:22.637103+0000","last_peered":"2024-02-21T23:25:22.637103+0000","last_clean":"2024-02-21T23:25:22.637103+0000","last_became_active":"2024-02-21T23:25:17.189302+0000","last_became_peered":"2024-02-21T23:25:17.189302+0000","last_unstale":"2024-02-21T23:25:22.637103+0000","last_undegraded":"2024-02-21T23:25:22.637103+0000","last_fullsized":"2024-02-21T23:25:22.637103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197389+0000","last_change":"2024-02-21T23:25:17.188113+0000","last_active":"2024-02-21T23:25:23.197389+0000","last_peered":"2024-02-21T23:25:23.197389+0000","last_clean":"2024-02-21T23:25:23.197389+0000","last_became_active":"2024-02-21T23:25:17.187811+0000","last_became_peered":"2024-02-21T23:25:17.187811+0000","last_unstale":"2024-02-21T23:25:23.197389+0000","last_undegraded":"2024-02-21T23:25:23.197389+0000","last_fullsized":"2024-02-21T23:25:23.197389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195329+0000","last_change":"2024-02-21T23:25:17.186415+0000","last_active":"2024-02-21T23:25:23.195329+0000","last_peered":"2024-02-21T23:25:23.195329+0000","last_clean":"2024-02-21T23:25:23.195329+0000","last_became_active":"2024-02-21T23:25:17.186021+0000","last_became_peered":"2024-02-21T23:25:17.186021+0000","last_unstale":"2024-02-21T23:25:23.195329+0000","last_undegraded":"2024-02-21T23:25:23.195329+0000","last_fullsized":"2024-02-21T23:25:23.195329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723114+0000","last_change":"2024-02-21T23:25:17.186378+0000","last_active":"2024-02-21T23:25:22.723114+0000","last_peered":"2024-02-21T23:25:22.723114+0000","last_clean":"2024-02-21T23:25:22.723114+0000","last_became_active":"2024-02-21T23:25:17.186086+0000","last_became_peered":"2024-02-21T23:25:17.186086+0000","last_unstale":"2024-02-21T23:25:22.723114+0000","last_undegraded":"2024-02-21T23:25:22.723114+0000","last_fullsized":"2024-02-21T23:25:22.723114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637345+0000","last_change":"2024-02-21T23:25:17.190934+0000","last_active":"2024-02-21T23:25:22.637345+0000","last_peered":"2024-02-21T23:25:22.637345+0000","last_clean":"2024-02-21T23:25:22.637345+0000","last_became_active":"2024-02-21T23:25:17.190680+0000","last_became_peered":"2024-02-21T23:25:17.190680+0000","last_unstale":"2024-02-21T23:25:22.637345+0000","last_undegraded":"2024-02-21T23:25:22.637345+0000","last_fullsized":"2024-02-21T23:25:22.637345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:22:04.238374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194866+0000","last_change":"2024-02-21T23:25:17.135140+0000","last_active":"2024-02-21T23:25:23.194866+0000","last_peered":"2024-02-21T23:25:23.194866+0000","last_clean":"2024-02-21T23:25:23.194866+0000","last_became_active":"2024-02-21T23:25:17.134866+0000","last_became_peered":"2024-02-21T23:25:17.134866+0000","last_unstale":"2024-02-21T23:25:23.194866+0000","last_undegraded":"2024-02-21T23:25:23.194866+0000","last_fullsized":"2024-02-21T23:25:23.194866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196657+0000","last_change":"2024-02-21T23:25:17.162204+0000","last_active":"2024-02-21T23:25:23.196657+0000","last_peered":"2024-02-21T23:25:23.196657+0000","last_clean":"2024-02-21T23:25:23.196657+0000","last_became_active":"2024-02-21T23:25:17.161807+0000","last_became_peered":"2024-02-21T23:25:17.161807+0000","last_unstale":"2024-02-21T23:25:23.196657+0000","last_undegraded":"2024-02-21T23:25:23.196657+0000","last_fullsized":"2024-02-21T23:25:23.196657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670677+0000","last_change":"2024-02-21T23:25:17.188932+0000","last_active":"2024-02-21T23:25:22.670677+0000","last_peered":"2024-02-21T23:25:22.670677+0000","last_clean":"2024-02-21T23:25:22.670677+0000","last_became_active":"2024-02-21T23:25:17.188646+0000","last_became_peered":"2024-02-21T23:25:17.188646+0000","last_unstale":"2024-02-21T23:25:22.670677+0000","last_undegraded":"2024-02-21T23:25:22.670677+0000","last_fullsized":"2024-02-21T23:25:22.670677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670892+0000","last_change":"2024-02-21T23:25:17.188333+0000","last_active":"2024-02-21T23:25:22.670892+0000","last_peered":"2024-02-21T23:25:22.670892+0000","last_clean":"2024-02-21T23:25:22.670892+0000","last_became_active":"2024-02-21T23:25:17.188066+0000","last_became_peered":"2024-02-21T23:25:17.188066+0000","last_unstale":"2024-02-21T23:25:22.670892+0000","last_undegraded":"2024-02-21T23:25:22.670892+0000","last_fullsized":"2024-02-21T23:25:22.670892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196168+0000","last_change":"2024-02-21T23:25:17.187549+0000","last_active":"2024-02-21T23:25:23.196168+0000","last_peered":"2024-02-21T23:25:23.196168+0000","last_clean":"2024-02-21T23:25:23.196168+0000","last_became_active":"2024-02-21T23:25:17.187297+0000","last_became_peered":"2024-02-21T23:25:17.187297+0000","last_unstale":"2024-02-21T23:25:23.196168+0000","last_undegraded":"2024-02-21T23:25:23.196168+0000","last_fullsized":"2024-02-21T23:25:23.196168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194774+0000","last_change":"2024-02-21T23:25:17.187352+0000","last_active":"2024-02-21T23:25:23.194774+0000","last_peered":"2024-02-21T23:25:23.194774+0000","last_clean":"2024-02-21T23:25:23.194774+0000","last_became_active":"2024-02-21T23:25:17.187156+0000","last_became_peered":"2024-02-21T23:25:17.187156+0000","last_unstale":"2024-02-21T23:25:23.194774+0000","last_undegraded":"2024-02-21T23:25:23.194774+0000","last_fullsized":"2024-02-21T23:25:23.194774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195829+0000","last_change":"2024-02-21T23:25:17.183264+0000","last_active":"2024-02-21T23:25:23.195829+0000","last_peered":"2024-02-21T23:25:23.195829+0000","last_clean":"2024-02-21T23:25:23.195829+0000","last_became_active":"2024-02-21T23:25:17.182924+0000","last_became_peered":"2024-02-21T23:25:17.182924+0000","last_unstale":"2024-02-21T23:25:23.195829+0000","last_undegraded":"2024-02-21T23:25:23.195829+0000","last_fullsized":"2024-02-21T23:25:23.195829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197389+0000","last_change":"2024-02-21T23:25:17.190841+0000","last_active":"2024-02-21T23:25:23.197389+0000","last_peered":"2024-02-21T23:25:23.197389+0000","last_clean":"2024-02-21T23:25:23.197389+0000","last_became_active":"2024-02-21T23:25:17.190276+0000","last_became_peered":"2024-02-21T23:25:17.190276+0000","last_unstale":"2024-02-21T23:25:23.197389+0000","last_undegraded":"2024-02-21T23:25:23.197389+0000","last_fullsized":"2024-02-21T23:25:23.197389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196058+0000","last_change":"2024-02-21T23:25:17.189872+0000","last_active":"2024-02-21T23:25:23.196058+0000","last_peered":"2024-02-21T23:25:23.196058+0000","last_clean":"2024-02-21T23:25:23.196058+0000","last_became_active":"2024-02-21T23:25:17.189614+0000","last_became_peered":"2024-02-21T23:25:17.189614+0000","last_unstale":"2024-02-21T23:25:23.196058+0000","last_undegraded":"2024-02-21T23:25:23.196058+0000","last_fullsized":"2024-02-21T23:25:23.196058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197136+0000","last_change":"2024-02-21T23:25:17.185771+0000","last_active":"2024-02-21T23:25:23.197136+0000","last_peered":"2024-02-21T23:25:23.197136+0000","last_clean":"2024-02-21T23:25:23.197136+0000","last_became_active":"2024-02-21T23:25:17.185458+0000","last_became_peered":"2024-02-21T23:25:17.185458+0000","last_unstale":"2024-02-21T23:25:23.197136+0000","last_undegraded":"2024-02-21T23:25:23.197136+0000","last_fullsized":"2024-02-21T23:25:23.197136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637050+0000","last_change":"2024-02-21T23:25:17.191159+0000","last_active":"2024-02-21T23:25:22.637050+0000","last_peered":"2024-02-21T23:25:22.637050+0000","last_clean":"2024-02-21T23:25:22.637050+0000","last_became_active":"2024-02-21T23:25:17.190576+0000","last_became_peered":"2024-02-21T23:25:17.190576+0000","last_unstale":"2024-02-21T23:25:22.637050+0000","last_undegraded":"2024-02-21T23:25:22.637050+0000","last_fullsized":"2024-02-21T23:25:22.637050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723892+0000","last_change":"2024-02-21T23:25:17.189287+0000","last_active":"2024-02-21T23:25:22.723892+0000","last_peered":"2024-02-21T23:25:22.723892+0000","last_clean":"2024-02-21T23:25:22.723892+0000","last_became_active":"2024-02-21T23:25:17.189029+0000","last_became_peered":"2024-02-21T23:25:17.189029+0000","last_unstale":"2024-02-21T23:25:22.723892+0000","last_undegraded":"2024-02-21T23:25:22.723892+0000","last_fullsized":"2024-02-21T23:25:22.723892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196058+0000","last_change":"2024-02-21T23:25:17.183737+0000","last_active":"2024-02-21T23:25:23.196058+0000","last_peered":"2024-02-21T23:25:23.196058+0000","last_clean":"2024-02-21T23:25:23.196058+0000","last_became_active":"2024-02-21T23:25:17.183410+0000","last_became_peered":"2024-02-21T23:25:17.183410+0000","last_unstale":"2024-02-21T23:25:23.196058+0000","last_undegraded":"2024-02-21T23:25:23.196058+0000","last_fullsized":"2024-02-21T23:25:23.196058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":691206,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":691262,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-21T23:25:51.437 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-21T23:25:51.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.588+0000 7f55f6f1e640 1 Processor -- start 2024-02-21T23:25:51.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.588+0000 7f55f6f1e640 1 -- start start 2024-02-21T23:25:51.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.588+0000 7f55f6f1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c5fd0 0x7f55f006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:51.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.588+0000 7f55f6f1e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f55f00c63b0 0x7f55f01580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:51.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.588+0000 7f55f6f1e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f55f01585e0 0x7f55f0151960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:51.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.588+0000 7f55f6f1e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f00cd460 con 0x7f55f00c63b0 2024-02-21T23:25:51.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.588+0000 7f55f6f1e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f0070b40 con 0x7f55f01585e0 2024-02-21T23:25:51.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.588+0000 7f55f6f1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00c7af0 con 0x7f55f00c5fd0 2024-02-21T23:25:51.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.589+0000 7f55effff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f55f00c63b0 0x7f55f01580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:51.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.589+0000 7f55f54bb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f55f01585e0 0x7f55f0151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:51.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.589+0000 7f55f4cba640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c5fd0 0x7f55f006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:51.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.590+0000 7f55effff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f55f00c63b0 0x7f55f01580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39910/0 (socket says 172.21.15.133:39910) 2024-02-21T23:25:51.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.590+0000 7f55effff640 1 -- 172.21.15.133:0/1113348806 learned_addr learned my addr 172.21.15.133:0/1113348806 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:51.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.592+0000 7f55f4cba640 1 -- 172.21.15.133:0/1113348806 >> v2:172.21.15.142:3300/0 conn(0x7f55f01585e0 msgr2=0x7f55f0151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:51.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.592+0000 7f55f4cba640 1 --2- 172.21.15.133:0/1113348806 >> v2:172.21.15.142:3300/0 conn(0x7f55f01585e0 0x7f55f0151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.592+0000 7f55f4cba640 1 -- 172.21.15.133:0/1113348806 >> v2:172.21.15.133:3300/0 conn(0x7f55f00c63b0 msgr2=0x7f55f01580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:51.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.592+0000 7f55f4cba640 1 --2- 172.21.15.133:0/1113348806 >> v2:172.21.15.133:3300/0 conn(0x7f55f00c63b0 0x7f55f01580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.592+0000 7f55f4cba640 1 -- 172.21.15.133:0/1113348806 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0158a50 con 0x7f55f00c5fd0 2024-02-21T23:25:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.594+0000 7f55f4cba640 1 --2- 172.21.15.133:0/1113348806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c5fd0 0x7f55f006b7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55f00ca520 tx=0x7f55e402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da193efb768a5767 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:51.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.598+0000 7f55ef7fe640 1 -- 172.21.15.133:0/1113348806 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4035020 con 0x7f55f00c5fd0 2024-02-21T23:25:51.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.598+0000 7f55ef7fe640 1 -- 172.21.15.133:0/1113348806 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e4031830 con 0x7f55f00c5fd0 2024-02-21T23:25:51.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.598+0000 7f55ef7fe640 1 -- 172.21.15.133:0/1113348806 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4031c00 con 0x7f55f00c5fd0 2024-02-21T23:25:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.599+0000 7f55f6f1e640 1 -- 172.21.15.133:0/1113348806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c5fd0 msgr2=0x7f55f006b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.599+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/1113348806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c5fd0 0x7f55f006b7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55f00ca520 tx=0x7f55e402b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.599+0000 7f55f6f1e640 1 -- 172.21.15.133:0/1113348806 shutdown_connections 2024-02-21T23:25:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.599+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/1113348806 >> v2:172.21.15.142:3300/0 conn(0x7f55f01585e0 0x7f55f0151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.599+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/1113348806 >> v2:172.21.15.133:3300/0 conn(0x7f55f00c63b0 0x7f55f01580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.599+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/1113348806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c5fd0 0x7f55f006b7a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.599+0000 7f55f6f1e640 1 -- 172.21.15.133:0/1113348806 >> 172.21.15.133:0/1113348806 conn(0x7f55f0061b50 msgr2=0x7f55f0072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:51.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.599+0000 7f55f6f1e640 1 -- 172.21.15.133:0/1113348806 shutdown_connections 2024-02-21T23:25:51.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.599+0000 7f55f6f1e640 1 -- 172.21.15.133:0/1113348806 wait complete. 2024-02-21T23:25:51.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.600+0000 7f55f6f1e640 1 Processor -- start 2024-02-21T23:25:51.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.601+0000 7f55f6f1e640 1 -- start start 2024-02-21T23:25:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.601+0000 7f55f6f1e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55f00c5fd0 0x7f55f0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:51.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.601+0000 7f55f6f1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c63b0 0x7f55f006c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:51.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.601+0000 7f55f6f1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f01585e0 0x7f55f01537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:51.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.601+0000 7f55f6f1e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00cce10 con 0x7f55f01585e0 2024-02-21T23:25:51.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.601+0000 7f55f6f1e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00ccb30 con 0x7f55f00c5fd0 2024-02-21T23:25:51.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.601+0000 7f55f6f1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00cd0f0 con 0x7f55f00c63b0 2024-02-21T23:25:51.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.602+0000 7f55f54bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f01585e0 0x7f55f01537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:51.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.602+0000 7f55effff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c63b0 0x7f55f006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:51.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.603+0000 7f55effff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c63b0 0x7f55f006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45658/0 (socket says 172.21.15.133:45658) 2024-02-21T23:25:51.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.603+0000 7f55effff640 1 -- 172.21.15.133:0/3162465266 learned_addr learned my addr 172.21.15.133:0/3162465266 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:51.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.603+0000 7f55f4cba640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55f00c5fd0 0x7f55f0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:51.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.606+0000 7f55effff640 1 -- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55f00c5fd0 msgr2=0x7f55f0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:51.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.606+0000 7f55effff640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55f00c5fd0 0x7f55f0155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.606+0000 7f55effff640 1 -- 172.21.15.133:0/3162465266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f01585e0 msgr2=0x7f55f01537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:51.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.606+0000 7f55effff640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f01585e0 0x7f55f01537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:51.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.606+0000 7f55effff640 1 -- 172.21.15.133:0/3162465266 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f01de530 con 0x7f55f00c63b0 2024-02-21T23:25:51.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.609+0000 7f55effff640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c63b0 0x7f55f006c240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00e8d0 tx=0x7f55dc00ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:51.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.612+0000 7f55ed7fa640 1 -- 172.21.15.133:0/3162465266 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc017070 con 0x7f55f00c63b0 2024-02-21T23:25:51.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.612+0000 7f55ed7fa640 1 -- 172.21.15.133:0/3162465266 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc00c940 con 0x7f55f00c63b0 2024-02-21T23:25:51.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.612+0000 7f55ed7fa640 1 -- 172.21.15.133:0/3162465266 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc00ccf0 con 0x7f55f00c63b0 2024-02-21T23:25:51.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.612+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f01de810 con 0x7f55f00c63b0 2024-02-21T23:25:51.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.612+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f01edd60 con 0x7f55f00c63b0 2024-02-21T23:25:51.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.613+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f006a130 con 0x7f55f00c63b0 2024-02-21T23:25:51.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.634+0000 7f55ed7fa640 1 -- 172.21.15.133:0/3162465266 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55dc01a070 con 0x7f55f00c63b0 2024-02-21T23:25:51.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.634+0000 7f55ed7fa640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f55d406b4d0 0x7f55d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:51.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.634+0000 7f55f4cba640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f55d406b4d0 0x7f55d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:51.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.635+0000 7f55f4cba640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f55d406b4d0 0x7f55d406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55e4036000 tx=0x7f55e4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:51.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.637+0000 7f55ed7fa640 1 -- 172.21.15.133:0/3162465266 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f55dc0571e0 con 0x7f55f00c63b0 2024-02-21T23:25:51.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:51.927+0000 7f55ed7fa640 1 -- 172.21.15.133:0/3162465266 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f55dc059140 con 0x7f55f00c63b0 2024-02-21T23:25:52.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.100+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55f0072260 con 0x7f55d406b4d0 2024-02-21T23:25:52.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.108+0000 7f55ed7fa640 1 -- 172.21.15.133:0/3162465266 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332612 (secure 0 0 0) 0x7f55f0072260 con 0x7f55d406b4d0 2024-02-21T23:25:52.118 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:52.119 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-02-21T23:25:52.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.111+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f55d406b4d0 msgr2=0x7f55d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.111+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f55d406b4d0 0x7f55d406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55e4036000 tx=0x7f55e4008030 comp rx=0 tx=0).stop 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.111+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c63b0 msgr2=0x7f55f006c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.111+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c63b0 0x7f55f006c240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00e8d0 tx=0x7f55dc00ed90 comp rx=0 tx=0).stop 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.111+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 shutdown_connections 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.112+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f55d406b4d0 0x7f55d406d990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.112+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f01585e0 0x7f55f01537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.112+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55f00c63b0 0x7f55f006c240 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.112+0000 7f55f6f1e640 1 --2- 172.21.15.133:0/3162465266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55f00c5fd0 0x7f55f0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.112+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 >> 172.21.15.133:0/3162465266 conn(0x7f55f0061b50 msgr2=0x7f55f014fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.112+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 shutdown_connections 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.112+0000 7f55f6f1e640 1 -- 172.21.15.133:0/3162465266 wait complete. 2024-02-21T23:25:52.122 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-02-21T23:25:50.691540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218332,"kb_used_data":2176,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756388,"statfs":{"total":773094113280,"available":772870541312,"internally_reserved":0,"allocated":2228224,"data_stored":1992064,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002481"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637827+0000","last_change":"2024-02-21T23:25:16.134390+0000","last_active":"2024-02-21T23:25:22.637827+0000","last_peered":"2024-02-21T23:25:22.637827+0000","last_clean":"2024-02-21T23:25:22.637827+0000","last_became_active":"2024-02-21T23:25:16.134112+0000","last_became_peered":"2024-02-21T23:25:16.134112+0000","last_unstale":"2024-02-21T23:25:22.637827+0000","last_undegraded":"2024-02-21T23:25:22.637827+0000","last_fullsized":"2024-02-21T23:25:22.637827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195309+0000","last_change":"2024-02-21T23:25:17.135738+0000","last_active":"2024-02-21T23:25:23.195309+0000","last_peered":"2024-02-21T23:25:23.195309+0000","last_clean":"2024-02-21T23:25:23.195309+0000","last_became_active":"2024-02-21T23:25:17.135586+0000","last_became_peered":"2024-02-21T23:25:17.135586+0000","last_unstale":"2024-02-21T23:25:23.195309+0000","last_undegraded":"2024-02-21T23:25:23.195309+0000","last_fullsized":"2024-02-21T23:25:23.195309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194873+0000","last_change":"2024-02-21T23:25:22.248962+0000","last_active":"2024-02-21T23:25:23.194873+0000","last_peered":"2024-02-21T23:25:23.194873+0000","last_clean":"2024-02-21T23:25:23.194873+0000","last_became_active":"2024-02-21T23:25:22.248610+0000","last_became_peered":"2024-02-21T23:25:22.248610+0000","last_unstale":"2024-02-21T23:25:23.194873+0000","last_undegraded":"2024-02-21T23:25:23.194873+0000","last_fullsized":"2024-02-21T23:25:23.194873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:45.939036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195369+0000","last_change":"2024-02-21T23:25:16.123635+0000","last_active":"2024-02-21T23:25:23.195369+0000","last_peered":"2024-02-21T23:25:23.195369+0000","last_clean":"2024-02-21T23:25:23.195369+0000","last_became_active":"2024-02-21T23:25:16.123374+0000","last_became_peered":"2024-02-21T23:25:16.123374+0000","last_unstale":"2024-02-21T23:25:23.195369+0000","last_undegraded":"2024-02-21T23:25:23.195369+0000","last_fullsized":"2024-02-21T23:25:23.195369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637647+0000","last_change":"2024-02-21T23:25:17.192858+0000","last_active":"2024-02-21T23:25:22.637647+0000","last_peered":"2024-02-21T23:25:22.637647+0000","last_clean":"2024-02-21T23:25:22.637647+0000","last_became_active":"2024-02-21T23:25:17.192626+0000","last_became_peered":"2024-02-21T23:25:17.192626+0000","last_unstale":"2024-02-21T23:25:22.637647+0000","last_undegraded":"2024-02-21T23:25:22.637647+0000","last_fullsized":"2024-02-21T23:25:22.637647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195812+0000","last_change":"2024-02-21T23:25:21.683490+0000","last_active":"2024-02-21T23:25:23.195812+0000","last_peered":"2024-02-21T23:25:23.195812+0000","last_clean":"2024-02-21T23:25:23.195812+0000","last_became_active":"2024-02-21T23:25:21.683173+0000","last_became_peered":"2024-02-21T23:25:21.683173+0000","last_unstale":"2024-02-21T23:25:23.195812+0000","last_undegraded":"2024-02-21T23:25:23.195812+0000","last_fullsized":"2024-02-21T23:25:23.195812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:49:01.803167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196175+0000","last_change":"2024-02-21T23:25:16.134805+0000","last_active":"2024-02-21T23:25:23.196175+0000","last_peered":"2024-02-21T23:25:23.196175+0000","last_clean":"2024-02-21T23:25:23.196175+0000","last_became_active":"2024-02-21T23:25:16.134533+0000","last_became_peered":"2024-02-21T23:25:16.134533+0000","last_unstale":"2024-02-21T23:25:23.196175+0000","last_undegraded":"2024-02-21T23:25:23.196175+0000","last_fullsized":"2024-02-21T23:25:23.196175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197229+0000","last_change":"2024-02-21T23:25:17.182131+0000","last_active":"2024-02-21T23:25:23.197229+0000","last_peered":"2024-02-21T23:25:23.197229+0000","last_clean":"2024-02-21T23:25:23.197229+0000","last_became_active":"2024-02-21T23:25:17.181886+0000","last_became_peered":"2024-02-21T23:25:17.181886+0000","last_unstale":"2024-02-21T23:25:23.197229+0000","last_undegraded":"2024-02-21T23:25:23.197229+0000","last_fullsized":"2024-02-21T23:25:23.197229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195843+0000","last_change":"2024-02-21T23:25:21.683506+0000","last_active":"2024-02-21T23:25:23.195843+0000","last_peered":"2024-02-21T23:25:23.195843+0000","last_clean":"2024-02-21T23:25:23.195843+0000","last_became_active":"2024-02-21T23:25:21.683115+0000","last_became_peered":"2024-02-21T23:25:21.683115+0000","last_unstale":"2024-02-21T23:25:23.195843+0000","last_undegraded":"2024-02-21T23:25:23.195843+0000","last_fullsized":"2024-02-21T23:25:23.195843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:13.865145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195380+0000","last_change":"2024-02-21T23:25:16.130851+0000","last_active":"2024-02-21T23:25:23.195380+0000","last_peered":"2024-02-21T23:25:23.195380+0000","last_clean":"2024-02-21T23:25:23.195380+0000","last_became_active":"2024-02-21T23:25:16.130604+0000","last_became_peered":"2024-02-21T23:25:16.130604+0000","last_unstale":"2024-02-21T23:25:23.195380+0000","last_undegraded":"2024-02-21T23:25:23.195380+0000","last_fullsized":"2024-02-21T23:25:23.195380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197689+0000","last_change":"2024-02-21T23:25:17.192642+0000","last_active":"2024-02-21T23:25:23.197689+0000","last_peered":"2024-02-21T23:25:23.197689+0000","last_clean":"2024-02-21T23:25:23.197689+0000","last_became_active":"2024-02-21T23:25:17.192394+0000","last_became_peered":"2024-02-21T23:25:17.192394+0000","last_unstale":"2024-02-21T23:25:23.197689+0000","last_undegraded":"2024-02-21T23:25:23.197689+0000","last_fullsized":"2024-02-21T23:25:23.197689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195564+0000","last_change":"2024-02-21T23:25:21.679491+0000","last_active":"2024-02-21T23:25:23.195564+0000","last_peered":"2024-02-21T23:25:23.195564+0000","last_clean":"2024-02-21T23:25:23.195564+0000","last_became_active":"2024-02-21T23:25:21.679117+0000","last_became_peered":"2024-02-21T23:25:21.679117+0000","last_unstale":"2024-02-21T23:25:23.195564+0000","last_undegraded":"2024-02-21T23:25:23.195564+0000","last_fullsized":"2024-02-21T23:25:23.195564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:52.680971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670455+0000","last_change":"2024-02-21T23:25:17.161438+0000","last_active":"2024-02-21T23:25:22.670455+0000","last_peered":"2024-02-21T23:25:22.670455+0000","last_clean":"2024-02-21T23:25:22.670455+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:25:22.670455+0000","last_undegraded":"2024-02-21T23:25:22.670455+0000","last_fullsized":"2024-02-21T23:25:22.670455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196550+0000","last_change":"2024-02-21T23:25:16.136913+0000","last_active":"2024-02-21T23:25:23.196550+0000","last_peered":"2024-02-21T23:25:23.196550+0000","last_clean":"2024-02-21T23:25:23.196550+0000","last_became_active":"2024-02-21T23:25:16.136628+0000","last_became_peered":"2024-02-21T23:25:16.136628+0000","last_unstale":"2024-02-21T23:25:23.196550+0000","last_undegraded":"2024-02-21T23:25:23.196550+0000","last_fullsized":"2024-02-21T23:25:23.196550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196588+0000","last_change":"2024-02-21T23:25:21.683706+0000","last_active":"2024-02-21T23:25:23.196588+0000","last_peered":"2024-02-21T23:25:23.196588+0000","last_clean":"2024-02-21T23:25:23.196588+0000","last_became_active":"2024-02-21T23:25:21.683116+0000","last_became_peered":"2024-02-21T23:25:21.683116+0000","last_unstale":"2024-02-21T23:25:23.196588+0000","last_undegraded":"2024-02-21T23:25:23.196588+0000","last_fullsized":"2024-02-21T23:25:23.196588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:06.265383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195016+0000","last_change":"2024-02-21T23:25:17.137829+0000","last_active":"2024-02-21T23:25:23.195016+0000","last_peered":"2024-02-21T23:25:23.195016+0000","last_clean":"2024-02-21T23:25:23.195016+0000","last_became_active":"2024-02-21T23:25:17.137557+0000","last_became_peered":"2024-02-21T23:25:17.137557+0000","last_unstale":"2024-02-21T23:25:23.195016+0000","last_undegraded":"2024-02-21T23:25:23.195016+0000","last_fullsized":"2024-02-21T23:25:23.195016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723448+0000","last_change":"2024-02-21T23:25:16.146181+0000","last_active":"2024-02-21T23:25:22.723448+0000","last_peered":"2024-02-21T23:25:22.723448+0000","last_clean":"2024-02-21T23:25:22.723448+0000","last_became_active":"2024-02-21T23:25:16.145932+0000","last_became_peered":"2024-02-21T23:25:16.145932+0000","last_unstale":"2024-02-21T23:25:22.723448+0000","last_undegraded":"2024-02-21T23:25:22.723448+0000","last_fullsized":"2024-02-21T23:25:22.723448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196544+0000","last_change":"2024-02-21T23:25:21.686044+0000","last_active":"2024-02-21T23:25:23.196544+0000","last_peered":"2024-02-21T23:25:23.196544+0000","last_clean":"2024-02-21T23:25:23.196544+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:25:23.196544+0000","last_undegraded":"2024-02-21T23:25:23.196544+0000","last_fullsized":"2024-02-21T23:25:23.196544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:39.740454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195078+0000","last_change":"2024-02-21T23:25:17.190236+0000","last_active":"2024-02-21T23:25:23.195078+0000","last_peered":"2024-02-21T23:25:23.195078+0000","last_clean":"2024-02-21T23:25:23.195078+0000","last_became_active":"2024-02-21T23:25:17.189957+0000","last_became_peered":"2024-02-21T23:25:17.189957+0000","last_unstale":"2024-02-21T23:25:23.195078+0000","last_undegraded":"2024-02-21T23:25:23.195078+0000","last_fullsized":"2024-02-21T23:25:23.195078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723142+0000","last_change":"2024-02-21T23:25:16.148108+0000","last_active":"2024-02-21T23:25:22.723142+0000","last_peered":"2024-02-21T23:25:22.723142+0000","last_clean":"2024-02-21T23:25:22.723142+0000","last_became_active":"2024-02-21T23:25:16.147930+0000","last_became_peered":"2024-02-21T23:25:16.147930+0000","last_unstale":"2024-02-21T23:25:22.723142+0000","last_undegraded":"2024-02-21T23:25:22.723142+0000","last_fullsized":"2024-02-21T23:25:22.723142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723970+0000","last_change":"2024-02-21T23:25:22.248150+0000","last_active":"2024-02-21T23:25:22.723970+0000","last_peered":"2024-02-21T23:25:22.723970+0000","last_clean":"2024-02-21T23:25:22.723970+0000","last_became_active":"2024-02-21T23:25:22.247769+0000","last_became_peered":"2024-02-21T23:25:22.247769+0000","last_unstale":"2024-02-21T23:25:22.723970+0000","last_undegraded":"2024-02-21T23:25:22.723970+0000","last_fullsized":"2024-02-21T23:25:22.723970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196252+0000","last_change":"2024-02-21T23:25:17.189758+0000","last_active":"2024-02-21T23:25:23.196252+0000","last_peered":"2024-02-21T23:25:23.196252+0000","last_clean":"2024-02-21T23:25:23.196252+0000","last_became_active":"2024-02-21T23:25:17.189492+0000","last_became_peered":"2024-02-21T23:25:17.189492+0000","last_unstale":"2024-02-21T23:25:23.196252+0000","last_undegraded":"2024-02-21T23:25:23.196252+0000","last_fullsized":"2024-02-21T23:25:23.196252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:42.526598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195319+0000","last_change":"2024-02-21T23:25:16.131611+0000","last_active":"2024-02-21T23:25:23.195319+0000","last_peered":"2024-02-21T23:25:23.195319+0000","last_clean":"2024-02-21T23:25:23.195319+0000","last_became_active":"2024-02-21T23:25:16.131158+0000","last_became_peered":"2024-02-21T23:25:16.131158+0000","last_unstale":"2024-02-21T23:25:23.195319+0000","last_undegraded":"2024-02-21T23:25:23.195319+0000","last_fullsized":"2024-02-21T23:25:23.195319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196480+0000","last_change":"2024-02-21T23:25:21.683821+0000","last_active":"2024-02-21T23:25:23.196480+0000","last_peered":"2024-02-21T23:25:23.196480+0000","last_clean":"2024-02-21T23:25:23.196480+0000","last_became_active":"2024-02-21T23:25:21.683466+0000","last_became_peered":"2024-02-21T23:25:21.683466+0000","last_unstale":"2024-02-21T23:25:23.196480+0000","last_undegraded":"2024-02-21T23:25:23.196480+0000","last_fullsized":"2024-02-21T23:25:23.196480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637653+0000","last_change":"2024-02-21T23:25:17.191394+0000","last_active":"2024-02-21T23:25:22.637653+0000","last_peered":"2024-02-21T23:25:22.637653+0000","last_clean":"2024-02-21T23:25:22.637653+0000","last_became_active":"2024-02-21T23:25:17.191106+0000","last_became_peered":"2024-02-21T23:25:17.191106+0000","last_unstale":"2024-02-21T23:25:22.637653+0000","last_undegraded":"2024-02-21T23:25:22.637653+0000","last_fullsized":"2024-02-21T23:25:22.637653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670984+0000","last_change":"2024-02-21T23:25:16.134862+0000","last_active":"2024-02-21T23:25:22.670984+0000","last_peered":"2024-02-21T23:25:22.670984+0000","last_clean":"2024-02-21T23:25:22.670984+0000","last_became_active":"2024-02-21T23:25:16.134602+0000","last_became_peered":"2024-02-21T23:25:16.134602+0000","last_unstale":"2024-02-21T23:25:22.670984+0000","last_undegraded":"2024-02-21T23:25:22.670984+0000","last_fullsized":"2024-02-21T23:25:22.670984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196150+0000","last_change":"2024-02-21T23:25:22.249606+0000","last_active":"2024-02-21T23:25:23.196150+0000","last_peered":"2024-02-21T23:25:23.196150+0000","last_clean":"2024-02-21T23:25:23.196150+0000","last_became_active":"2024-02-21T23:25:22.249214+0000","last_became_peered":"2024-02-21T23:25:22.249214+0000","last_unstale":"2024-02-21T23:25:23.196150+0000","last_undegraded":"2024-02-21T23:25:23.196150+0000","last_fullsized":"2024-02-21T23:25:23.196150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:36.929076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195603+0000","last_change":"2024-02-21T23:25:17.183290+0000","last_active":"2024-02-21T23:25:23.195603+0000","last_peered":"2024-02-21T23:25:23.195603+0000","last_clean":"2024-02-21T23:25:23.195603+0000","last_became_active":"2024-02-21T23:25:17.183047+0000","last_became_peered":"2024-02-21T23:25:17.183047+0000","last_unstale":"2024-02-21T23:25:23.195603+0000","last_undegraded":"2024-02-21T23:25:23.195603+0000","last_fullsized":"2024-02-21T23:25:23.195603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723977+0000","last_change":"2024-02-21T23:25:16.146202+0000","last_active":"2024-02-21T23:25:22.723977+0000","last_peered":"2024-02-21T23:25:22.723977+0000","last_clean":"2024-02-21T23:25:22.723977+0000","last_became_active":"2024-02-21T23:25:16.145911+0000","last_became_peered":"2024-02-21T23:25:16.145911+0000","last_unstale":"2024-02-21T23:25:22.723977+0000","last_undegraded":"2024-02-21T23:25:22.723977+0000","last_fullsized":"2024-02-21T23:25:22.723977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194768+0000","last_change":"2024-02-21T23:25:22.243297+0000","last_active":"2024-02-21T23:25:23.194768+0000","last_peered":"2024-02-21T23:25:23.194768+0000","last_clean":"2024-02-21T23:25:23.194768+0000","last_became_active":"2024-02-21T23:25:22.242922+0000","last_became_peered":"2024-02-21T23:25:22.242922+0000","last_unstale":"2024-02-21T23:25:23.194768+0000","last_undegraded":"2024-02-21T23:25:23.194768+0000","last_fullsized":"2024-02-21T23:25:23.194768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:52:51.773574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723650+0000","last_change":"2024-02-21T23:25:17.190577+0000","last_active":"2024-02-21T23:25:22.723650+0000","last_peered":"2024-02-21T23:25:22.723650+0000","last_clean":"2024-02-21T23:25:22.723650+0000","last_became_active":"2024-02-21T23:25:17.190332+0000","last_became_peered":"2024-02-21T23:25:17.190332+0000","last_unstale":"2024-02-21T23:25:22.723650+0000","last_undegraded":"2024-02-21T23:25:22.723650+0000","last_fullsized":"2024-02-21T23:25:22.723650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196263+0000","last_change":"2024-02-21T23:25:16.134076+0000","last_active":"2024-02-21T23:25:23.196263+0000","last_peered":"2024-02-21T23:25:23.196263+0000","last_clean":"2024-02-21T23:25:23.196263+0000","last_became_active":"2024-02-21T23:25:16.133816+0000","last_became_peered":"2024-02-21T23:25:16.133816+0000","last_unstale":"2024-02-21T23:25:23.196263+0000","last_undegraded":"2024-02-21T23:25:23.196263+0000","last_fullsized":"2024-02-21T23:25:23.196263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637332+0000","last_change":"2024-02-21T23:25:22.243360+0000","last_active":"2024-02-21T23:25:22.637332+0000","last_peered":"2024-02-21T23:25:22.637332+0000","last_clean":"2024-02-21T23:25:22.637332+0000","last_became_active":"2024-02-21T23:25:22.242958+0000","last_became_peered":"2024-02-21T23:25:22.242958+0000","last_unstale":"2024-02-21T23:25:22.637332+0000","last_undegraded":"2024-02-21T23:25:22.637332+0000","last_fullsized":"2024-02-21T23:25:22.637332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197769+0000","last_change":"2024-02-21T23:25:17.192402+0000","last_active":"2024-02-21T23:25:23.197769+0000","last_peered":"2024-02-21T23:25:23.197769+0000","last_clean":"2024-02-21T23:25:23.197769+0000","last_became_active":"2024-02-21T23:25:17.192138+0000","last_became_peered":"2024-02-21T23:25:17.192138+0000","last_unstale":"2024-02-21T23:25:23.197769+0000","last_undegraded":"2024-02-21T23:25:23.197769+0000","last_fullsized":"2024-02-21T23:25:23.197769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723671+0000","last_change":"2024-02-21T23:25:16.145521+0000","last_active":"2024-02-21T23:25:22.723671+0000","last_peered":"2024-02-21T23:25:22.723671+0000","last_clean":"2024-02-21T23:25:22.723671+0000","last_became_active":"2024-02-21T23:25:16.145255+0000","last_became_peered":"2024-02-21T23:25:16.145255+0000","last_unstale":"2024-02-21T23:25:22.723671+0000","last_undegraded":"2024-02-21T23:25:22.723671+0000","last_fullsized":"2024-02-21T23:25:22.723671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723238+0000","last_change":"2024-02-21T23:25:21.683849+0000","last_active":"2024-02-21T23:25:22.723238+0000","last_peered":"2024-02-21T23:25:22.723238+0000","last_clean":"2024-02-21T23:25:22.723238+0000","last_became_active":"2024-02-21T23:25:21.683379+0000","last_became_peered":"2024-02-21T23:25:21.683379+0000","last_unstale":"2024-02-21T23:25:22.723238+0000","last_undegraded":"2024-02-21T23:25:22.723238+0000","last_fullsized":"2024-02-21T23:25:22.723238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196877+0000","last_change":"2024-02-21T23:25:17.166232+0000","last_active":"2024-02-21T23:25:23.196877+0000","last_peered":"2024-02-21T23:25:23.196877+0000","last_clean":"2024-02-21T23:25:23.196877+0000","last_became_active":"2024-02-21T23:25:17.165999+0000","last_became_peered":"2024-02-21T23:25:17.165999+0000","last_unstale":"2024-02-21T23:25:23.196877+0000","last_undegraded":"2024-02-21T23:25:23.196877+0000","last_fullsized":"2024-02-21T23:25:23.196877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:45.125986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197593+0000","last_change":"2024-02-21T23:25:16.147082+0000","last_active":"2024-02-21T23:25:23.197593+0000","last_peered":"2024-02-21T23:25:23.197593+0000","last_clean":"2024-02-21T23:25:23.197593+0000","last_became_active":"2024-02-21T23:25:16.146797+0000","last_became_peered":"2024-02-21T23:25:16.146797+0000","last_unstale":"2024-02-21T23:25:23.197593+0000","last_undegraded":"2024-02-21T23:25:23.197593+0000","last_fullsized":"2024-02-21T23:25:23.197593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723612+0000","last_change":"2024-02-21T23:25:21.682860+0000","last_active":"2024-02-21T23:25:22.723612+0000","last_peered":"2024-02-21T23:25:22.723612+0000","last_clean":"2024-02-21T23:25:22.723612+0000","last_became_active":"2024-02-21T23:25:21.682404+0000","last_became_peered":"2024-02-21T23:25:21.682404+0000","last_unstale":"2024-02-21T23:25:22.723612+0000","last_undegraded":"2024-02-21T23:25:22.723612+0000","last_fullsized":"2024-02-21T23:25:22.723612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723262+0000","last_change":"2024-02-21T23:25:17.190425+0000","last_active":"2024-02-21T23:25:22.723262+0000","last_peered":"2024-02-21T23:25:22.723262+0000","last_clean":"2024-02-21T23:25:22.723262+0000","last_became_active":"2024-02-21T23:25:17.190172+0000","last_became_peered":"2024-02-21T23:25:17.190172+0000","last_unstale":"2024-02-21T23:25:22.723262+0000","last_undegraded":"2024-02-21T23:25:22.723262+0000","last_fullsized":"2024-02-21T23:25:22.723262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636897+0000","last_change":"2024-02-21T23:25:16.108263+0000","last_active":"2024-02-21T23:25:22.636897+0000","last_peered":"2024-02-21T23:25:22.636897+0000","last_clean":"2024-02-21T23:25:22.636897+0000","last_became_active":"2024-02-21T23:25:16.107822+0000","last_became_peered":"2024-02-21T23:25:16.107822+0000","last_unstale":"2024-02-21T23:25:22.636897+0000","last_undegraded":"2024-02-21T23:25:22.636897+0000","last_fullsized":"2024-02-21T23:25:22.636897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723957+0000","last_change":"2024-02-21T23:25:21.686034+0000","last_active":"2024-02-21T23:25:22.723957+0000","last_peered":"2024-02-21T23:25:22.723957+0000","last_clean":"2024-02-21T23:25:22.723957+0000","last_became_active":"2024-02-21T23:25:21.685546+0000","last_became_peered":"2024-02-21T23:25:21.685546+0000","last_unstale":"2024-02-21T23:25:22.723957+0000","last_undegraded":"2024-02-21T23:25:22.723957+0000","last_fullsized":"2024-02-21T23:25:22.723957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636614+0000","last_change":"2024-02-21T23:25:17.161738+0000","last_active":"2024-02-21T23:25:22.636614+0000","last_peered":"2024-02-21T23:25:22.636614+0000","last_clean":"2024-02-21T23:25:22.636614+0000","last_became_active":"2024-02-21T23:25:17.161459+0000","last_became_peered":"2024-02-21T23:25:17.161459+0000","last_unstale":"2024-02-21T23:25:22.636614+0000","last_undegraded":"2024-02-21T23:25:22.636614+0000","last_fullsized":"2024-02-21T23:25:22.636614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:31:58.434561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195408+0000","last_change":"2024-02-21T23:25:16.147162+0000","last_active":"2024-02-21T23:25:23.195408+0000","last_peered":"2024-02-21T23:25:23.195408+0000","last_clean":"2024-02-21T23:25:23.195408+0000","last_became_active":"2024-02-21T23:25:16.146850+0000","last_became_peered":"2024-02-21T23:25:16.146850+0000","last_unstale":"2024-02-21T23:25:23.195408+0000","last_undegraded":"2024-02-21T23:25:23.195408+0000","last_fullsized":"2024-02-21T23:25:23.195408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637976+0000","last_change":"2024-02-21T23:25:22.248109+0000","last_active":"2024-02-21T23:25:22.637976+0000","last_peered":"2024-02-21T23:25:22.637976+0000","last_clean":"2024-02-21T23:25:22.637976+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:25:22.637976+0000","last_undegraded":"2024-02-21T23:25:22.637976+0000","last_fullsized":"2024-02-21T23:25:22.637976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194934+0000","last_change":"2024-02-21T23:25:17.192450+0000","last_active":"2024-02-21T23:25:23.194934+0000","last_peered":"2024-02-21T23:25:23.194934+0000","last_clean":"2024-02-21T23:25:23.194934+0000","last_became_active":"2024-02-21T23:25:17.192167+0000","last_became_peered":"2024-02-21T23:25:17.192167+0000","last_unstale":"2024-02-21T23:25:23.194934+0000","last_undegraded":"2024-02-21T23:25:23.194934+0000","last_fullsized":"2024-02-21T23:25:23.194934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197488+0000","last_change":"2024-02-21T23:25:16.127688+0000","last_active":"2024-02-21T23:25:23.197488+0000","last_peered":"2024-02-21T23:25:23.197488+0000","last_clean":"2024-02-21T23:25:23.197488+0000","last_became_active":"2024-02-21T23:25:16.127388+0000","last_became_peered":"2024-02-21T23:25:16.127388+0000","last_unstale":"2024-02-21T23:25:23.197488+0000","last_undegraded":"2024-02-21T23:25:23.197488+0000","last_fullsized":"2024-02-21T23:25:23.197488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195483+0000","last_change":"2024-02-21T23:25:05.612817+0000","last_active":"2024-02-21T23:25:23.195483+0000","last_peered":"2024-02-21T23:25:23.195483+0000","last_clean":"2024-02-21T23:25:23.195483+0000","last_became_active":"2024-02-21T23:25:05.612483+0000","last_became_peered":"2024-02-21T23:25:05.612483+0000","last_unstale":"2024-02-21T23:25:23.195483+0000","last_undegraded":"2024-02-21T23:25:23.195483+0000","last_fullsized":"2024-02-21T23:25:23.195483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_clean_scrub_stamp":"2024-02-21T23:25:03.736732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:23:33.073040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670664+0000","last_change":"2024-02-21T23:25:22.248174+0000","last_active":"2024-02-21T23:25:22.670664+0000","last_peered":"2024-02-21T23:25:22.670664+0000","last_clean":"2024-02-21T23:25:22.670664+0000","last_became_active":"2024-02-21T23:25:22.247834+0000","last_became_peered":"2024-02-21T23:25:22.247834+0000","last_unstale":"2024-02-21T23:25:22.670664+0000","last_undegraded":"2024-02-21T23:25:22.670664+0000","last_fullsized":"2024-02-21T23:25:22.670664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:13.953358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196092+0000","last_change":"2024-02-21T23:25:17.192575+0000","last_active":"2024-02-21T23:25:23.196092+0000","last_peered":"2024-02-21T23:25:23.196092+0000","last_clean":"2024-02-21T23:25:23.196092+0000","last_became_active":"2024-02-21T23:25:17.192293+0000","last_became_peered":"2024-02-21T23:25:17.192293+0000","last_unstale":"2024-02-21T23:25:23.196092+0000","last_undegraded":"2024-02-21T23:25:23.196092+0000","last_fullsized":"2024-02-21T23:25:23.196092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637073+0000","last_change":"2024-02-21T23:25:16.128369+0000","last_active":"2024-02-21T23:25:22.637073+0000","last_peered":"2024-02-21T23:25:22.637073+0000","last_clean":"2024-02-21T23:25:22.637073+0000","last_became_active":"2024-02-21T23:25:16.128067+0000","last_became_peered":"2024-02-21T23:25:16.128067+0000","last_unstale":"2024-02-21T23:25:22.637073+0000","last_undegraded":"2024-02-21T23:25:22.637073+0000","last_fullsized":"2024-02-21T23:25:22.637073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670470+0000","last_change":"2024-02-21T23:25:22.243282+0000","last_active":"2024-02-21T23:25:22.670470+0000","last_peered":"2024-02-21T23:25:22.670470+0000","last_clean":"2024-02-21T23:25:22.670470+0000","last_became_active":"2024-02-21T23:25:22.242916+0000","last_became_peered":"2024-02-21T23:25:22.242916+0000","last_unstale":"2024-02-21T23:25:22.670470+0000","last_undegraded":"2024-02-21T23:25:22.670470+0000","last_fullsized":"2024-02-21T23:25:22.670470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197322+0000","last_change":"2024-02-21T23:25:17.185418+0000","last_active":"2024-02-21T23:25:23.197322+0000","last_peered":"2024-02-21T23:25:23.197322+0000","last_clean":"2024-02-21T23:25:23.197322+0000","last_became_active":"2024-02-21T23:25:17.185119+0000","last_became_peered":"2024-02-21T23:25:17.185119+0000","last_unstale":"2024-02-21T23:25:23.197322+0000","last_undegraded":"2024-02-21T23:25:23.197322+0000","last_fullsized":"2024-02-21T23:25:23.197322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723756+0000","last_change":"2024-02-21T23:25:16.151088+0000","last_active":"2024-02-21T23:25:22.723756+0000","last_peered":"2024-02-21T23:25:22.723756+0000","last_clean":"2024-02-21T23:25:22.723756+0000","last_became_active":"2024-02-21T23:25:16.150819+0000","last_became_peered":"2024-02-21T23:25:16.150819+0000","last_unstale":"2024-02-21T23:25:22.723756+0000","last_undegraded":"2024-02-21T23:25:22.723756+0000","last_fullsized":"2024-02-21T23:25:22.723756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637420+0000","last_change":"2024-02-21T23:25:21.684592+0000","last_active":"2024-02-21T23:25:22.637420+0000","last_peered":"2024-02-21T23:25:22.637420+0000","last_clean":"2024-02-21T23:25:22.637420+0000","last_became_active":"2024-02-21T23:25:21.684192+0000","last_became_peered":"2024-02-21T23:25:21.684192+0000","last_unstale":"2024-02-21T23:25:22.637420+0000","last_undegraded":"2024-02-21T23:25:22.637420+0000","last_fullsized":"2024-02-21T23:25:22.637420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196248+0000","last_change":"2024-02-21T23:25:17.184268+0000","last_active":"2024-02-21T23:25:23.196248+0000","last_peered":"2024-02-21T23:25:23.196248+0000","last_clean":"2024-02-21T23:25:23.196248+0000","last_became_active":"2024-02-21T23:25:17.184027+0000","last_became_peered":"2024-02-21T23:25:17.184027+0000","last_unstale":"2024-02-21T23:25:23.196248+0000","last_undegraded":"2024-02-21T23:25:23.196248+0000","last_fullsized":"2024-02-21T23:25:23.196248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637140+0000","last_change":"2024-02-21T23:25:16.108099+0000","last_active":"2024-02-21T23:25:22.637140+0000","last_peered":"2024-02-21T23:25:22.637140+0000","last_clean":"2024-02-21T23:25:22.637140+0000","last_became_active":"2024-02-21T23:25:16.107815+0000","last_became_peered":"2024-02-21T23:25:16.107815+0000","last_unstale":"2024-02-21T23:25:22.637140+0000","last_undegraded":"2024-02-21T23:25:22.637140+0000","last_fullsized":"2024-02-21T23:25:22.637140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195214+0000","last_change":"2024-02-21T23:25:22.248926+0000","last_active":"2024-02-21T23:25:23.195214+0000","last_peered":"2024-02-21T23:25:23.195214+0000","last_clean":"2024-02-21T23:25:23.195214+0000","last_became_active":"2024-02-21T23:25:22.248558+0000","last_became_peered":"2024-02-21T23:25:22.248558+0000","last_unstale":"2024-02-21T23:25:23.195214+0000","last_undegraded":"2024-02-21T23:25:23.195214+0000","last_fullsized":"2024-02-21T23:25:23.195214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195709+0000","last_change":"2024-02-21T23:25:17.183254+0000","last_active":"2024-02-21T23:25:23.195709+0000","last_peered":"2024-02-21T23:25:23.195709+0000","last_clean":"2024-02-21T23:25:23.195709+0000","last_became_active":"2024-02-21T23:25:17.183020+0000","last_became_peered":"2024-02-21T23:25:17.183020+0000","last_unstale":"2024-02-21T23:25:23.195709+0000","last_undegraded":"2024-02-21T23:25:23.195709+0000","last_fullsized":"2024-02-21T23:25:23.195709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196256+0000","last_change":"2024-02-21T23:25:16.129131+0000","last_active":"2024-02-21T23:25:23.196256+0000","last_peered":"2024-02-21T23:25:23.196256+0000","last_clean":"2024-02-21T23:25:23.196256+0000","last_became_active":"2024-02-21T23:25:16.128853+0000","last_became_peered":"2024-02-21T23:25:16.128853+0000","last_unstale":"2024-02-21T23:25:23.196256+0000","last_undegraded":"2024-02-21T23:25:23.196256+0000","last_fullsized":"2024-02-21T23:25:23.196256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:29.302181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195886+0000","last_change":"2024-02-21T23:25:21.684342+0000","last_active":"2024-02-21T23:25:23.195886+0000","last_peered":"2024-02-21T23:25:23.195886+0000","last_clean":"2024-02-21T23:25:23.195886+0000","last_became_active":"2024-02-21T23:25:21.683991+0000","last_became_peered":"2024-02-21T23:25:21.683991+0000","last_unstale":"2024-02-21T23:25:23.195886+0000","last_undegraded":"2024-02-21T23:25:23.195886+0000","last_fullsized":"2024-02-21T23:25:23.195886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:21:00.312314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195905+0000","last_change":"2024-02-21T23:25:17.183442+0000","last_active":"2024-02-21T23:25:23.195905+0000","last_peered":"2024-02-21T23:25:23.195905+0000","last_clean":"2024-02-21T23:25:23.195905+0000","last_became_active":"2024-02-21T23:25:17.183057+0000","last_became_peered":"2024-02-21T23:25:17.183057+0000","last_unstale":"2024-02-21T23:25:23.195905+0000","last_undegraded":"2024-02-21T23:25:23.195905+0000","last_fullsized":"2024-02-21T23:25:23.195905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636480+0000","last_change":"2024-02-21T23:25:16.145801+0000","last_active":"2024-02-21T23:25:22.636480+0000","last_peered":"2024-02-21T23:25:22.636480+0000","last_clean":"2024-02-21T23:25:22.636480+0000","last_became_active":"2024-02-21T23:25:16.145523+0000","last_became_peered":"2024-02-21T23:25:16.145523+0000","last_unstale":"2024-02-21T23:25:22.636480+0000","last_undegraded":"2024-02-21T23:25:22.636480+0000","last_fullsized":"2024-02-21T23:25:22.636480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637169+0000","last_change":"2024-02-21T23:25:21.679191+0000","last_active":"2024-02-21T23:25:22.637169+0000","last_peered":"2024-02-21T23:25:22.637169+0000","last_clean":"2024-02-21T23:25:22.637169+0000","last_became_active":"2024-02-21T23:25:21.678715+0000","last_became_peered":"2024-02-21T23:25:21.678715+0000","last_unstale":"2024-02-21T23:25:22.637169+0000","last_undegraded":"2024-02-21T23:25:22.637169+0000","last_fullsized":"2024-02-21T23:25:22.637169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670230+0000","last_change":"2024-02-21T23:25:17.138401+0000","last_active":"2024-02-21T23:25:22.670230+0000","last_peered":"2024-02-21T23:25:22.670230+0000","last_clean":"2024-02-21T23:25:22.670230+0000","last_became_active":"2024-02-21T23:25:17.138136+0000","last_became_peered":"2024-02-21T23:25:17.138136+0000","last_unstale":"2024-02-21T23:25:22.670230+0000","last_undegraded":"2024-02-21T23:25:22.670230+0000","last_fullsized":"2024-02-21T23:25:22.670230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670228+0000","last_change":"2024-02-21T23:25:16.130209+0000","last_active":"2024-02-21T23:25:22.670228+0000","last_peered":"2024-02-21T23:25:22.670228+0000","last_clean":"2024-02-21T23:25:22.670228+0000","last_became_active":"2024-02-21T23:25:16.129974+0000","last_became_peered":"2024-02-21T23:25:16.129974+0000","last_unstale":"2024-02-21T23:25:22.670228+0000","last_undegraded":"2024-02-21T23:25:22.670228+0000","last_fullsized":"2024-02-21T23:25:22.670228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195559+0000","last_change":"2024-02-21T23:25:22.243303+0000","last_active":"2024-02-21T23:25:23.195559+0000","last_peered":"2024-02-21T23:25:23.195559+0000","last_clean":"2024-02-21T23:25:23.195559+0000","last_became_active":"2024-02-21T23:25:22.242960+0000","last_became_peered":"2024-02-21T23:25:22.242960+0000","last_unstale":"2024-02-21T23:25:23.195559+0000","last_undegraded":"2024-02-21T23:25:23.195559+0000","last_fullsized":"2024-02-21T23:25:23.195559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:54:16.386354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196403+0000","last_change":"2024-02-21T23:25:17.184233+0000","last_active":"2024-02-21T23:25:23.196403+0000","last_peered":"2024-02-21T23:25:23.196403+0000","last_clean":"2024-02-21T23:25:23.196403+0000","last_became_active":"2024-02-21T23:25:17.183980+0000","last_became_peered":"2024-02-21T23:25:17.183980+0000","last_unstale":"2024-02-21T23:25:23.196403+0000","last_undegraded":"2024-02-21T23:25:23.196403+0000","last_fullsized":"2024-02-21T23:25:23.196403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197566+0000","last_change":"2024-02-21T23:25:16.131147+0000","last_active":"2024-02-21T23:25:23.197566+0000","last_peered":"2024-02-21T23:25:23.197566+0000","last_clean":"2024-02-21T23:25:23.197566+0000","last_became_active":"2024-02-21T23:25:16.130553+0000","last_became_peered":"2024-02-21T23:25:16.130553+0000","last_unstale":"2024-02-21T23:25:23.197566+0000","last_undegraded":"2024-02-21T23:25:23.197566+0000","last_fullsized":"2024-02-21T23:25:23.197566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195579+0000","last_change":"2024-02-21T23:25:22.243261+0000","last_active":"2024-02-21T23:25:23.195579+0000","last_peered":"2024-02-21T23:25:23.195579+0000","last_clean":"2024-02-21T23:25:23.195579+0000","last_became_active":"2024-02-21T23:25:22.242900+0000","last_became_peered":"2024-02-21T23:25:22.242900+0000","last_unstale":"2024-02-21T23:25:23.195579+0000","last_undegraded":"2024-02-21T23:25:23.195579+0000","last_fullsized":"2024-02-21T23:25:23.195579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:34:44.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637437+0000","last_change":"2024-02-21T23:25:17.183303+0000","last_active":"2024-02-21T23:25:22.637437+0000","last_peered":"2024-02-21T23:25:22.637437+0000","last_clean":"2024-02-21T23:25:22.637437+0000","last_became_active":"2024-02-21T23:25:17.183042+0000","last_became_peered":"2024-02-21T23:25:17.183042+0000","last_unstale":"2024-02-21T23:25:22.637437+0000","last_undegraded":"2024-02-21T23:25:22.637437+0000","last_fullsized":"2024-02-21T23:25:22.637437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637466+0000","last_change":"2024-02-21T23:25:16.127609+0000","last_active":"2024-02-21T23:25:22.637466+0000","last_peered":"2024-02-21T23:25:22.637466+0000","last_clean":"2024-02-21T23:25:22.637466+0000","last_became_active":"2024-02-21T23:25:16.127375+0000","last_became_peered":"2024-02-21T23:25:16.127375+0000","last_unstale":"2024-02-21T23:25:22.637466+0000","last_undegraded":"2024-02-21T23:25:22.637466+0000","last_fullsized":"2024-02-21T23:25:22.637466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197762+0000","last_change":"2024-02-21T23:25:22.248329+0000","last_active":"2024-02-21T23:25:23.197762+0000","last_peered":"2024-02-21T23:25:23.197762+0000","last_clean":"2024-02-21T23:25:23.197762+0000","last_became_active":"2024-02-21T23:25:22.247889+0000","last_became_peered":"2024-02-21T23:25:22.247889+0000","last_unstale":"2024-02-21T23:25:23.197762+0000","last_undegraded":"2024-02-21T23:25:23.197762+0000","last_fullsized":"2024-02-21T23:25:23.197762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196057+0000","last_change":"2024-02-21T23:25:17.183741+0000","last_active":"2024-02-21T23:25:23.196057+0000","last_peered":"2024-02-21T23:25:23.196057+0000","last_clean":"2024-02-21T23:25:23.196057+0000","last_became_active":"2024-02-21T23:25:17.183408+0000","last_became_peered":"2024-02-21T23:25:17.183408+0000","last_unstale":"2024-02-21T23:25:23.196057+0000","last_undegraded":"2024-02-21T23:25:23.196057+0000","last_fullsized":"2024-02-21T23:25:23.196057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:38.383407+0000","last_change":"2024-02-21T23:25:16.116460+0000","last_active":"2024-02-21T23:25:38.383407+0000","last_peered":"2024-02-21T23:25:38.383407+0000","last_clean":"2024-02-21T23:25:38.383407+0000","last_became_active":"2024-02-21T23:25:16.116162+0000","last_became_peered":"2024-02-21T23:25:16.116162+0000","last_unstale":"2024-02-21T23:25:38.383407+0000","last_undegraded":"2024-02-21T23:25:38.383407+0000","last_fullsized":"2024-02-21T23:25:38.383407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723404+0000","last_change":"2024-02-21T23:25:21.683199+0000","last_active":"2024-02-21T23:25:22.723404+0000","last_peered":"2024-02-21T23:25:22.723404+0000","last_clean":"2024-02-21T23:25:22.723404+0000","last_became_active":"2024-02-21T23:25:21.682425+0000","last_became_peered":"2024-02-21T23:25:21.682425+0000","last_unstale":"2024-02-21T23:25:22.723404+0000","last_undegraded":"2024-02-21T23:25:22.723404+0000","last_fullsized":"2024-02-21T23:25:22.723404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196728+0000","last_change":"2024-02-21T23:25:17.184837+0000","last_active":"2024-02-21T23:25:23.196728+0000","last_peered":"2024-02-21T23:25:23.196728+0000","last_clean":"2024-02-21T23:25:23.196728+0000","last_became_active":"2024-02-21T23:25:17.184574+0000","last_became_peered":"2024-02-21T23:25:17.184574+0000","last_unstale":"2024-02-21T23:25:23.196728+0000","last_undegraded":"2024-02-21T23:25:23.196728+0000","last_fullsized":"2024-02-21T23:25:23.196728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723604+0000","last_change":"2024-02-21T23:25:16.151031+0000","last_active":"2024-02-21T23:25:22.723604+0000","last_peered":"2024-02-21T23:25:22.723604+0000","last_clean":"2024-02-21T23:25:22.723604+0000","last_became_active":"2024-02-21T23:25:16.150767+0000","last_became_peered":"2024-02-21T23:25:16.150767+0000","last_unstale":"2024-02-21T23:25:22.723604+0000","last_undegraded":"2024-02-21T23:25:22.723604+0000","last_fullsized":"2024-02-21T23:25:22.723604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196772+0000","last_change":"2024-02-21T23:25:21.683995+0000","last_active":"2024-02-21T23:25:23.196772+0000","last_peered":"2024-02-21T23:25:23.196772+0000","last_clean":"2024-02-21T23:25:23.196772+0000","last_became_active":"2024-02-21T23:25:21.683415+0000","last_became_peered":"2024-02-21T23:25:21.683415+0000","last_unstale":"2024-02-21T23:25:23.196772+0000","last_undegraded":"2024-02-21T23:25:23.196772+0000","last_fullsized":"2024-02-21T23:25:23.196772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637736+0000","last_change":"2024-02-21T23:25:17.187230+0000","last_active":"2024-02-21T23:25:22.637736+0000","last_peered":"2024-02-21T23:25:22.637736+0000","last_clean":"2024-02-21T23:25:22.637736+0000","last_became_active":"2024-02-21T23:25:17.186952+0000","last_became_peered":"2024-02-21T23:25:17.186952+0000","last_unstale":"2024-02-21T23:25:22.637736+0000","last_undegraded":"2024-02-21T23:25:22.637736+0000","last_fullsized":"2024-02-21T23:25:22.637736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197393+0000","last_change":"2024-02-21T23:25:16.134722+0000","last_active":"2024-02-21T23:25:23.197393+0000","last_peered":"2024-02-21T23:25:23.197393+0000","last_clean":"2024-02-21T23:25:23.197393+0000","last_became_active":"2024-02-21T23:25:16.134458+0000","last_became_peered":"2024-02-21T23:25:16.134458+0000","last_unstale":"2024-02-21T23:25:23.197393+0000","last_undegraded":"2024-02-21T23:25:23.197393+0000","last_fullsized":"2024-02-21T23:25:23.197393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195006+0000","last_change":"2024-02-21T23:25:21.684109+0000","last_active":"2024-02-21T23:25:23.195006+0000","last_peered":"2024-02-21T23:25:23.195006+0000","last_clean":"2024-02-21T23:25:23.195006+0000","last_became_active":"2024-02-21T23:25:21.683705+0000","last_became_peered":"2024-02-21T23:25:21.683705+0000","last_unstale":"2024-02-21T23:25:23.195006+0000","last_undegraded":"2024-02-21T23:25:23.195006+0000","last_fullsized":"2024-02-21T23:25:23.195006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197237+0000","last_change":"2024-02-21T23:25:17.188374+0000","last_active":"2024-02-21T23:25:23.197237+0000","last_peered":"2024-02-21T23:25:23.197237+0000","last_clean":"2024-02-21T23:25:23.197237+0000","last_became_active":"2024-02-21T23:25:17.187792+0000","last_became_peered":"2024-02-21T23:25:17.187792+0000","last_unstale":"2024-02-21T23:25:23.197237+0000","last_undegraded":"2024-02-21T23:25:23.197237+0000","last_fullsized":"2024-02-21T23:25:23.197237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670904+0000","last_change":"2024-02-21T23:25:16.129446+0000","last_active":"2024-02-21T23:25:22.670904+0000","last_peered":"2024-02-21T23:25:22.670904+0000","last_clean":"2024-02-21T23:25:22.670904+0000","last_became_active":"2024-02-21T23:25:16.129164+0000","last_became_peered":"2024-02-21T23:25:16.129164+0000","last_unstale":"2024-02-21T23:25:22.670904+0000","last_undegraded":"2024-02-21T23:25:22.670904+0000","last_fullsized":"2024-02-21T23:25:22.670904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194890+0000","last_change":"2024-02-21T23:25:22.248096+0000","last_active":"2024-02-21T23:25:23.194890+0000","last_peered":"2024-02-21T23:25:23.194890+0000","last_clean":"2024-02-21T23:25:23.194890+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:25:23.194890+0000","last_undegraded":"2024-02-21T23:25:23.194890+0000","last_fullsized":"2024-02-21T23:25:23.194890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637006+0000","last_change":"2024-02-21T23:25:22.248119+0000","last_active":"2024-02-21T23:25:22.637006+0000","last_peered":"2024-02-21T23:25:22.637006+0000","last_clean":"2024-02-21T23:25:22.637006+0000","last_became_active":"2024-02-21T23:25:22.247811+0000","last_became_peered":"2024-02-21T23:25:22.247811+0000","last_unstale":"2024-02-21T23:25:22.637006+0000","last_undegraded":"2024-02-21T23:25:22.637006+0000","last_fullsized":"2024-02-21T23:25:22.637006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723036+0000","last_change":"2024-02-21T23:25:17.189176+0000","last_active":"2024-02-21T23:25:22.723036+0000","last_peered":"2024-02-21T23:25:22.723036+0000","last_clean":"2024-02-21T23:25:22.723036+0000","last_became_active":"2024-02-21T23:25:17.188885+0000","last_became_peered":"2024-02-21T23:25:17.188885+0000","last_unstale":"2024-02-21T23:25:22.723036+0000","last_undegraded":"2024-02-21T23:25:22.723036+0000","last_fullsized":"2024-02-21T23:25:22.723036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195502+0000","last_change":"2024-02-21T23:25:16.132036+0000","last_active":"2024-02-21T23:25:23.195502+0000","last_peered":"2024-02-21T23:25:23.195502+0000","last_clean":"2024-02-21T23:25:23.195502+0000","last_became_active":"2024-02-21T23:25:16.131756+0000","last_became_peered":"2024-02-21T23:25:16.131756+0000","last_unstale":"2024-02-21T23:25:23.195502+0000","last_undegraded":"2024-02-21T23:25:23.195502+0000","last_fullsized":"2024-02-21T23:25:23.195502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670902+0000","last_change":"2024-02-21T23:25:21.686133+0000","last_active":"2024-02-21T23:25:22.670902+0000","last_peered":"2024-02-21T23:25:22.670902+0000","last_clean":"2024-02-21T23:25:22.670902+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:25:22.670902+0000","last_undegraded":"2024-02-21T23:25:22.670902+0000","last_fullsized":"2024-02-21T23:25:22.670902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194875+0000","last_change":"2024-02-21T23:25:17.135158+0000","last_active":"2024-02-21T23:25:23.194875+0000","last_peered":"2024-02-21T23:25:23.194875+0000","last_clean":"2024-02-21T23:25:23.194875+0000","last_became_active":"2024-02-21T23:25:17.134888+0000","last_became_peered":"2024-02-21T23:25:17.134888+0000","last_unstale":"2024-02-21T23:25:23.194875+0000","last_undegraded":"2024-02-21T23:25:23.194875+0000","last_fullsized":"2024-02-21T23:25:23.194875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194948+0000","last_change":"2024-02-21T23:25:16.116378+0000","last_active":"2024-02-21T23:25:23.194948+0000","last_peered":"2024-02-21T23:25:23.194948+0000","last_clean":"2024-02-21T23:25:23.194948+0000","last_became_active":"2024-02-21T23:25:16.116201+0000","last_became_peered":"2024-02-21T23:25:16.116201+0000","last_unstale":"2024-02-21T23:25:23.194948+0000","last_undegraded":"2024-02-21T23:25:23.194948+0000","last_fullsized":"2024-02-21T23:25:23.194948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670541+0000","last_change":"2024-02-21T23:25:16.132314+0000","last_active":"2024-02-21T23:25:22.670541+0000","last_peered":"2024-02-21T23:25:22.670541+0000","last_clean":"2024-02-21T23:25:22.670541+0000","last_became_active":"2024-02-21T23:25:16.132062+0000","last_became_peered":"2024-02-21T23:25:16.132062+0000","last_unstale":"2024-02-21T23:25:22.670541+0000","last_undegraded":"2024-02-21T23:25:22.670541+0000","last_fullsized":"2024-02-21T23:25:22.670541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637458+0000","last_change":"2024-02-21T23:25:17.161451+0000","last_active":"2024-02-21T23:25:22.637458+0000","last_peered":"2024-02-21T23:25:22.637458+0000","last_clean":"2024-02-21T23:25:22.637458+0000","last_became_active":"2024-02-21T23:25:17.161176+0000","last_became_peered":"2024-02-21T23:25:17.161176+0000","last_unstale":"2024-02-21T23:25:22.637458+0000","last_undegraded":"2024-02-21T23:25:22.637458+0000","last_fullsized":"2024-02-21T23:25:22.637458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636872+0000","last_change":"2024-02-21T23:25:21.685724+0000","last_active":"2024-02-21T23:25:22.636872+0000","last_peered":"2024-02-21T23:25:22.636872+0000","last_clean":"2024-02-21T23:25:22.636872+0000","last_became_active":"2024-02-21T23:25:21.685375+0000","last_became_peered":"2024-02-21T23:25:21.685375+0000","last_unstale":"2024-02-21T23:25:22.636872+0000","last_undegraded":"2024-02-21T23:25:22.636872+0000","last_fullsized":"2024-02-21T23:25:22.636872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:56:28.795401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194995+0000","last_change":"2024-02-21T23:25:16.134127+0000","last_active":"2024-02-21T23:25:23.194995+0000","last_peered":"2024-02-21T23:25:23.194995+0000","last_clean":"2024-02-21T23:25:23.194995+0000","last_became_active":"2024-02-21T23:25:16.133839+0000","last_became_peered":"2024-02-21T23:25:16.133839+0000","last_unstale":"2024-02-21T23:25:23.194995+0000","last_undegraded":"2024-02-21T23:25:23.194995+0000","last_fullsized":"2024-02-21T23:25:23.194995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195317+0000","last_change":"2024-02-21T23:25:17.190455+0000","last_active":"2024-02-21T23:25:23.195317+0000","last_peered":"2024-02-21T23:25:23.195317+0000","last_clean":"2024-02-21T23:25:23.195317+0000","last_became_active":"2024-02-21T23:25:17.190194+0000","last_became_peered":"2024-02-21T23:25:17.190194+0000","last_unstale":"2024-02-21T23:25:23.195317+0000","last_undegraded":"2024-02-21T23:25:23.195317+0000","last_fullsized":"2024-02-21T23:25:23.195317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:03.451409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195261+0000","last_change":"2024-02-21T23:25:21.683501+0000","last_active":"2024-02-21T23:25:23.195261+0000","last_peered":"2024-02-21T23:25:23.195261+0000","last_clean":"2024-02-21T23:25:23.195261+0000","last_became_active":"2024-02-21T23:25:21.683139+0000","last_became_peered":"2024-02-21T23:25:21.683139+0000","last_unstale":"2024-02-21T23:25:23.195261+0000","last_undegraded":"2024-02-21T23:25:23.195261+0000","last_fullsized":"2024-02-21T23:25:23.195261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-22T23:58:15.072875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670359+0000","last_change":"2024-02-21T23:25:17.138039+0000","last_active":"2024-02-21T23:25:22.670359+0000","last_peered":"2024-02-21T23:25:22.670359+0000","last_clean":"2024-02-21T23:25:22.670359+0000","last_became_active":"2024-02-21T23:25:17.137717+0000","last_became_peered":"2024-02-21T23:25:17.137717+0000","last_unstale":"2024-02-21T23:25:22.670359+0000","last_undegraded":"2024-02-21T23:25:22.670359+0000","last_fullsized":"2024-02-21T23:25:22.670359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636758+0000","last_change":"2024-02-21T23:25:17.194727+0000","last_active":"2024-02-21T23:25:22.636758+0000","last_peered":"2024-02-21T23:25:22.636758+0000","last_clean":"2024-02-21T23:25:22.636758+0000","last_became_active":"2024-02-21T23:25:17.194533+0000","last_became_peered":"2024-02-21T23:25:17.194533+0000","last_unstale":"2024-02-21T23:25:22.636758+0000","last_undegraded":"2024-02-21T23:25:22.636758+0000","last_fullsized":"2024-02-21T23:25:22.636758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197507+0000","last_change":"2024-02-21T23:25:17.191051+0000","last_active":"2024-02-21T23:25:23.197507+0000","last_peered":"2024-02-21T23:25:23.197507+0000","last_clean":"2024-02-21T23:25:23.197507+0000","last_became_active":"2024-02-21T23:25:17.190600+0000","last_became_peered":"2024-02-21T23:25:17.190600+0000","last_unstale":"2024-02-21T23:25:23.197507+0000","last_undegraded":"2024-02-21T23:25:23.197507+0000","last_fullsized":"2024-02-21T23:25:23.197507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637377+0000","last_change":"2024-02-21T23:25:17.135311+0000","last_active":"2024-02-21T23:25:22.637377+0000","last_peered":"2024-02-21T23:25:22.637377+0000","last_clean":"2024-02-21T23:25:22.637377+0000","last_became_active":"2024-02-21T23:25:17.135011+0000","last_became_peered":"2024-02-21T23:25:17.135011+0000","last_unstale":"2024-02-21T23:25:22.637377+0000","last_undegraded":"2024-02-21T23:25:22.637377+0000","last_fullsized":"2024-02-21T23:25:22.637377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195670+0000","last_change":"2024-02-21T23:25:17.161696+0000","last_active":"2024-02-21T23:25:23.195670+0000","last_peered":"2024-02-21T23:25:23.195670+0000","last_clean":"2024-02-21T23:25:23.195670+0000","last_became_active":"2024-02-21T23:25:17.161401+0000","last_became_peered":"2024-02-21T23:25:17.161401+0000","last_unstale":"2024-02-21T23:25:23.195670+0000","last_undegraded":"2024-02-21T23:25:23.195670+0000","last_fullsized":"2024-02-21T23:25:23.195670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723507+0000","last_change":"2024-02-21T23:25:17.189731+0000","last_active":"2024-02-21T23:25:22.723507+0000","last_peered":"2024-02-21T23:25:22.723507+0000","last_clean":"2024-02-21T23:25:22.723507+0000","last_became_active":"2024-02-21T23:25:17.189484+0000","last_became_peered":"2024-02-21T23:25:17.189484+0000","last_unstale":"2024-02-21T23:25:22.723507+0000","last_undegraded":"2024-02-21T23:25:22.723507+0000","last_fullsized":"2024-02-21T23:25:22.723507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670963+0000","last_change":"2024-02-21T23:25:17.184125+0000","last_active":"2024-02-21T23:25:22.670963+0000","last_peered":"2024-02-21T23:25:22.670963+0000","last_clean":"2024-02-21T23:25:22.670963+0000","last_became_active":"2024-02-21T23:25:17.183865+0000","last_became_peered":"2024-02-21T23:25:17.183865+0000","last_unstale":"2024-02-21T23:25:22.670963+0000","last_undegraded":"2024-02-21T23:25:22.670963+0000","last_fullsized":"2024-02-21T23:25:22.670963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195515+0000","last_change":"2024-02-21T23:25:17.161482+0000","last_active":"2024-02-21T23:25:23.195515+0000","last_peered":"2024-02-21T23:25:23.195515+0000","last_clean":"2024-02-21T23:25:23.195515+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:25:23.195515+0000","last_undegraded":"2024-02-21T23:25:23.195515+0000","last_fullsized":"2024-02-21T23:25:23.195515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195414+0000","last_change":"2024-02-21T23:25:17.185691+0000","last_active":"2024-02-21T23:25:23.195414+0000","last_peered":"2024-02-21T23:25:23.195414+0000","last_clean":"2024-02-21T23:25:23.195414+0000","last_became_active":"2024-02-21T23:25:17.185420+0000","last_became_peered":"2024-02-21T23:25:17.185420+0000","last_unstale":"2024-02-21T23:25:23.195414+0000","last_undegraded":"2024-02-21T23:25:23.195414+0000","last_fullsized":"2024-02-21T23:25:23.195414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194975+0000","last_change":"2024-02-21T23:25:17.161321+0000","last_active":"2024-02-21T23:25:23.194975+0000","last_peered":"2024-02-21T23:25:23.194975+0000","last_clean":"2024-02-21T23:25:23.194975+0000","last_became_active":"2024-02-21T23:25:17.161010+0000","last_became_peered":"2024-02-21T23:25:17.161010+0000","last_unstale":"2024-02-21T23:25:23.194975+0000","last_undegraded":"2024-02-21T23:25:23.194975+0000","last_fullsized":"2024-02-21T23:25:23.194975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197503+0000","last_change":"2024-02-21T23:25:17.191072+0000","last_active":"2024-02-21T23:25:23.197503+0000","last_peered":"2024-02-21T23:25:23.197503+0000","last_clean":"2024-02-21T23:25:23.197503+0000","last_became_active":"2024-02-21T23:25:17.187741+0000","last_became_peered":"2024-02-21T23:25:17.187741+0000","last_unstale":"2024-02-21T23:25:23.197503+0000","last_undegraded":"2024-02-21T23:25:23.197503+0000","last_fullsized":"2024-02-21T23:25:23.197503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637229+0000","last_change":"2024-02-21T23:25:17.186616+0000","last_active":"2024-02-21T23:25:22.637229+0000","last_peered":"2024-02-21T23:25:22.637229+0000","last_clean":"2024-02-21T23:25:22.637229+0000","last_became_active":"2024-02-21T23:25:17.186108+0000","last_became_peered":"2024-02-21T23:25:17.186108+0000","last_unstale":"2024-02-21T23:25:22.637229+0000","last_undegraded":"2024-02-21T23:25:22.637229+0000","last_fullsized":"2024-02-21T23:25:22.637229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670481+0000","last_change":"2024-02-21T23:25:17.161428+0000","last_active":"2024-02-21T23:25:22.670481+0000","last_peered":"2024-02-21T23:25:22.670481+0000","last_clean":"2024-02-21T23:25:22.670481+0000","last_became_active":"2024-02-21T23:25:17.161140+0000","last_became_peered":"2024-02-21T23:25:17.161140+0000","last_unstale":"2024-02-21T23:25:22.670481+0000","last_undegraded":"2024-02-21T23:25:22.670481+0000","last_fullsized":"2024-02-21T23:25:22.670481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196229+0000","last_change":"2024-02-21T23:25:17.190666+0000","last_active":"2024-02-21T23:25:23.196229+0000","last_peered":"2024-02-21T23:25:23.196229+0000","last_clean":"2024-02-21T23:25:23.196229+0000","last_became_active":"2024-02-21T23:25:17.190420+0000","last_became_peered":"2024-02-21T23:25:17.190420+0000","last_unstale":"2024-02-21T23:25:23.196229+0000","last_undegraded":"2024-02-21T23:25:23.196229+0000","last_fullsized":"2024-02-21T23:25:23.196229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637103+0000","last_change":"2024-02-21T23:25:17.189583+0000","last_active":"2024-02-21T23:25:22.637103+0000","last_peered":"2024-02-21T23:25:22.637103+0000","last_clean":"2024-02-21T23:25:22.637103+0000","last_became_active":"2024-02-21T23:25:17.189302+0000","last_became_peered":"2024-02-21T23:25:17.189302+0000","last_unstale":"2024-02-21T23:25:22.637103+0000","last_undegraded":"2024-02-21T23:25:22.637103+0000","last_fullsized":"2024-02-21T23:25:22.637103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197389+0000","last_change":"2024-02-21T23:25:17.188113+0000","last_active":"2024-02-21T23:25:23.197389+0000","last_peered":"2024-02-21T23:25:23.197389+0000","last_clean":"2024-02-21T23:25:23.197389+0000","last_became_active":"2024-02-21T23:25:17.187811+0000","last_became_peered":"2024-02-21T23:25:17.187811+0000","last_unstale":"2024-02-21T23:25:23.197389+0000","last_undegraded":"2024-02-21T23:25:23.197389+0000","last_fullsized":"2024-02-21T23:25:23.197389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195329+0000","last_change":"2024-02-21T23:25:17.186415+0000","last_active":"2024-02-21T23:25:23.195329+0000","last_peered":"2024-02-21T23:25:23.195329+0000","last_clean":"2024-02-21T23:25:23.195329+0000","last_became_active":"2024-02-21T23:25:17.186021+0000","last_became_peered":"2024-02-21T23:25:17.186021+0000","last_unstale":"2024-02-21T23:25:23.195329+0000","last_undegraded":"2024-02-21T23:25:23.195329+0000","last_fullsized":"2024-02-21T23:25:23.195329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723114+0000","last_change":"2024-02-21T23:25:17.186378+0000","last_active":"2024-02-21T23:25:22.723114+0000","last_peered":"2024-02-21T23:25:22.723114+0000","last_clean":"2024-02-21T23:25:22.723114+0000","last_became_active":"2024-02-21T23:25:17.186086+0000","last_became_peered":"2024-02-21T23:25:17.186086+0000","last_unstale":"2024-02-21T23:25:22.723114+0000","last_undegraded":"2024-02-21T23:25:22.723114+0000","last_fullsized":"2024-02-21T23:25:22.723114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637345+0000","last_change":"2024-02-21T23:25:17.190934+0000","last_active":"2024-02-21T23:25:22.637345+0000","last_peered":"2024-02-21T23:25:22.637345+0000","last_clean":"2024-02-21T23:25:22.637345+0000","last_became_active":"2024-02-21T23:25:17.190680+0000","last_became_peered":"2024-02-21T23:25:17.190680+0000","last_unstale":"2024-02-21T23:25:22.637345+0000","last_undegraded":"2024-02-21T23:25:22.637345+0000","last_fullsized":"2024-02-21T23:25:22.637345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:22:04.238374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194866+0000","last_change":"2024-02-21T23:25:17.135140+0000","last_active":"2024-02-21T23:25:23.194866+0000","last_peered":"2024-02-21T23:25:23.194866+0000","last_clean":"2024-02-21T23:25:23.194866+0000","last_became_active":"2024-02-21T23:25:17.134866+0000","last_became_peered":"2024-02-21T23:25:17.134866+0000","last_unstale":"2024-02-21T23:25:23.194866+0000","last_undegraded":"2024-02-21T23:25:23.194866+0000","last_fullsized":"2024-02-21T23:25:23.194866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196657+0000","last_change":"2024-02-21T23:25:17.162204+0000","last_active":"2024-02-21T23:25:23.196657+0000","last_peered":"2024-02-21T23:25:23.196657+0000","last_clean":"2024-02-21T23:25:23.196657+0000","last_became_active":"2024-02-21T23:25:17.161807+0000","last_became_peered":"2024-02-21T23:25:17.161807+0000","last_unstale":"2024-02-21T23:25:23.196657+0000","last_undegraded":"2024-02-21T23:25:23.196657+0000","last_fullsized":"2024-02-21T23:25:23.196657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670677+0000","last_change":"2024-02-21T23:25:17.188932+0000","last_active":"2024-02-21T23:25:22.670677+0000","last_peered":"2024-02-21T23:25:22.670677+0000","last_clean":"2024-02-21T23:25:22.670677+0000","last_became_active":"2024-02-21T23:25:17.188646+0000","last_became_peered":"2024-02-21T23:25:17.188646+0000","last_unstale":"2024-02-21T23:25:22.670677+0000","last_undegraded":"2024-02-21T23:25:22.670677+0000","last_fullsized":"2024-02-21T23:25:22.670677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670892+0000","last_change":"2024-02-21T23:25:17.188333+0000","last_active":"2024-02-21T23:25:22.670892+0000","last_peered":"2024-02-21T23:25:22.670892+0000","last_clean":"2024-02-21T23:25:22.670892+0000","last_became_active":"2024-02-21T23:25:17.188066+0000","last_became_peered":"2024-02-21T23:25:17.188066+0000","last_unstale":"2024-02-21T23:25:22.670892+0000","last_undegraded":"2024-02-21T23:25:22.670892+0000","last_fullsized":"2024-02-21T23:25:22.670892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196168+0000","last_change":"2024-02-21T23:25:17.187549+0000","last_active":"2024-02-21T23:25:23.196168+0000","last_peered":"2024-02-21T23:25:23.196168+0000","last_clean":"2024-02-21T23:25:23.196168+0000","last_became_active":"2024-02-21T23:25:17.187297+0000","last_became_peered":"2024-02-21T23:25:17.187297+0000","last_unstale":"2024-02-21T23:25:23.196168+0000","last_undegraded":"2024-02-21T23:25:23.196168+0000","last_fullsized":"2024-02-21T23:25:23.196168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194774+0000","last_change":"2024-02-21T23:25:17.187352+0000","last_active":"2024-02-21T23:25:23.194774+0000","last_peered":"2024-02-21T23:25:23.194774+0000","last_clean":"2024-02-21T23:25:23.194774+0000","last_became_active":"2024-02-21T23:25:17.187156+0000","last_became_peered":"2024-02-21T23:25:17.187156+0000","last_unstale":"2024-02-21T23:25:23.194774+0000","last_undegraded":"2024-02-21T23:25:23.194774+0000","last_fullsized":"2024-02-21T23:25:23.194774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195829+0000","last_change":"2024-02-21T23:25:17.183264+0000","last_active":"2024-02-21T23:25:23.195829+0000","last_peered":"2024-02-21T23:25:23.195829+0000","last_clean":"2024-02-21T23:25:23.195829+0000","last_became_active":"2024-02-21T23:25:17.182924+0000","last_became_peered":"2024-02-21T23:25:17.182924+0000","last_unstale":"2024-02-21T23:25:23.195829+0000","last_undegraded":"2024-02-21T23:25:23.195829+0000","last_fullsized":"2024-02-21T23:25:23.195829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197389+0000","last_change":"2024-02-21T23:25:17.190841+0000","last_active":"2024-02-21T23:25:23.197389+0000","last_peered":"2024-02-21T23:25:23.197389+0000","last_clean":"2024-02-21T23:25:23.197389+0000","last_became_active":"2024-02-21T23:25:17.190276+0000","last_became_peered":"2024-02-21T23:25:17.190276+0000","last_unstale":"2024-02-21T23:25:23.197389+0000","last_undegraded":"2024-02-21T23:25:23.197389+0000","last_fullsized":"2024-02-21T23:25:23.197389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196058+0000","last_change":"2024-02-21T23:25:17.189872+0000","last_active":"2024-02-21T23:25:23.196058+0000","last_peered":"2024-02-21T23:25:23.196058+0000","last_clean":"2024-02-21T23:25:23.196058+0000","last_became_active":"2024-02-21T23:25:17.189614+0000","last_became_peered":"2024-02-21T23:25:17.189614+0000","last_unstale":"2024-02-21T23:25:23.196058+0000","last_undegraded":"2024-02-21T23:25:23.196058+0000","last_fullsized":"2024-02-21T23:25:23.196058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197136+0000","last_change":"2024-02-21T23:25:17.185771+0000","last_active":"2024-02-21T23:25:23.197136+0000","last_peered":"2024-02-21T23:25:23.197136+0000","last_clean":"2024-02-21T23:25:23.197136+0000","last_became_active":"2024-02-21T23:25:17.185458+0000","last_became_peered":"2024-02-21T23:25:17.185458+0000","last_unstale":"2024-02-21T23:25:23.197136+0000","last_undegraded":"2024-02-21T23:25:23.197136+0000","last_fullsized":"2024-02-21T23:25:23.197136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637050+0000","last_change":"2024-02-21T23:25:17.191159+0000","last_active":"2024-02-21T23:25:22.637050+0000","last_peered":"2024-02-21T23:25:22.637050+0000","last_clean":"2024-02-21T23:25:22.637050+0000","last_became_active":"2024-02-21T23:25:17.190576+0000","last_became_peered":"2024-02-21T23:25:17.190576+0000","last_unstale":"2024-02-21T23:25:22.637050+0000","last_undegraded":"2024-02-21T23:25:22.637050+0000","last_fullsized":"2024-02-21T23:25:22.637050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723892+0000","last_change":"2024-02-21T23:25:17.189287+0000","last_active":"2024-02-21T23:25:22.723892+0000","last_peered":"2024-02-21T23:25:22.723892+0000","last_clean":"2024-02-21T23:25:22.723892+0000","last_became_active":"2024-02-21T23:25:17.189029+0000","last_became_peered":"2024-02-21T23:25:17.189029+0000","last_unstale":"2024-02-21T23:25:22.723892+0000","last_undegraded":"2024-02-21T23:25:22.723892+0000","last_fullsized":"2024-02-21T23:25:22.723892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196058+0000","last_change":"2024-02-21T23:25:17.183737+0000","last_active":"2024-02-21T23:25:23.196058+0000","last_peered":"2024-02-21T23:25:23.196058+0000","last_clean":"2024-02-21T23:25:23.196058+0000","last_became_active":"2024-02-21T23:25:17.183410+0000","last_became_peered":"2024-02-21T23:25:17.183410+0000","last_unstale":"2024-02-21T23:25:23.196058+0000","last_undegraded":"2024-02-21T23:25:23.196058+0000","last_fullsized":"2024-02-21T23:25:23.196058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":691206,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":691262,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-21T23:25:52.127 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-21T23:25:52.127 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-21T23:25:52.127 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-02-21T23:25:52.128 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-02-21T23:25:52.128 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-02-21T23:25:52.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.244+0000 7efd97f08640 1 Processor -- start 2024-02-21T23:25:52.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.244+0000 7efd97f08640 1 -- start start 2024-02-21T23:25:52.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.244+0000 7efd97f08640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900cdfe0 0x7efd900c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.244+0000 7efd97f08640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd900ce3c0 0x7efd90070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.244+0000 7efd97f08640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd900717e0 con 0x7efd900712b0 2024-02-21T23:25:52.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.245+0000 7efd97f08640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd900c84b0 con 0x7efd900ce3c0 2024-02-21T23:25:52.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.245+0000 7efd97f08640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd90158ca0 con 0x7efd900cdfe0 2024-02-21T23:25:52.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.245+0000 7efd954a3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd900ce3c0 0x7efd90070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.246+0000 7efd954a3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd900ce3c0 0x7efd90070d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39924/0 (socket says 172.21.15.133:39924) 2024-02-21T23:25:52.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.246+0000 7efd954a3640 1 -- 172.21.15.133:0/1453791947 learned_addr learned my addr 172.21.15.133:0/1453791947 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:52.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.247+0000 7efd95ca4640 1 --2- 172.21.15.133:0/1453791947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900cdfe0 0x7efd900c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.248+0000 7efd954a3640 1 -- 172.21.15.133:0/1453791947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900cdfe0 msgr2=0x7efd900c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:52.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.248+0000 7efd954a3640 1 --2- 172.21.15.133:0/1453791947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900cdfe0 0x7efd900c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.248+0000 7efd954a3640 1 -- 172.21.15.133:0/1453791947 >> v1:172.21.15.142:6789/0 conn(0x7efd900712b0 legacy=0x7efd9014e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.248+0000 7efd954a3640 1 -- 172.21.15.133:0/1453791947 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd90158090 con 0x7efd900ce3c0 2024-02-21T23:25:52.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.250+0000 7efd954a3640 1 --2- 172.21.15.133:0/1453791947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900ce3c0 0x7efd90070d70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd90158e20 tx=0x7efd8002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2204d385cda31b48 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:52.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.253+0000 7efd94ca2640 1 -- 172.21.15.133:0/1453791947 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80034070 con 0x7efd900ce3c0 2024-02-21T23:25:52.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.253+0000 7efd94ca2640 1 -- 172.21.15.133:0/1453791947 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd80030840 con 0x7efd900ce3c0 2024-02-21T23:25:52.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.253+0000 7efd94ca2640 1 -- 172.21.15.133:0/1453791947 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80030c10 con 0x7efd900ce3c0 2024-02-21T23:25:52.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.254+0000 7efd97f08640 1 -- 172.21.15.133:0/1453791947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900ce3c0 msgr2=0x7efd90070d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.254+0000 7efd97f08640 1 --2- 172.21.15.133:0/1453791947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900ce3c0 0x7efd90070d70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd90158e20 tx=0x7efd8002b980 comp rx=0 tx=0).stop 2024-02-21T23:25:52.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.254+0000 7efd97f08640 1 -- 172.21.15.133:0/1453791947 shutdown_connections 2024-02-21T23:25:52.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.254+0000 7efd97f08640 1 --2- 172.21.15.133:0/1453791947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900ce3c0 0x7efd90070d70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.254+0000 7efd97f08640 1 --2- 172.21.15.133:0/1453791947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900cdfe0 0x7efd900c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.254+0000 7efd97f08640 1 -- 172.21.15.133:0/1453791947 >> 172.21.15.133:0/1453791947 conn(0x7efd9006aec0 msgr2=0x7efd9006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:52.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.254+0000 7efd97f08640 1 -- 172.21.15.133:0/1453791947 shutdown_connections 2024-02-21T23:25:52.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.254+0000 7efd97f08640 1 -- 172.21.15.133:0/1453791947 wait complete. 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.255+0000 7efd97f08640 1 Processor -- start 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.255+0000 7efd97f08640 1 -- start start 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.256+0000 7efd97f08640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900712b0 0x7efd9015ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.256+0000 7efd97f08640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd900cdfe0 0x7efd900c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.256+0000 7efd97f08640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900ce3c0 0x7efd901f0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.256+0000 7efd97f08640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd90158930 con 0x7efd900712b0 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.256+0000 7efd97f08640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd90158650 con 0x7efd900cdfe0 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.256+0000 7efd97f08640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9014ec20 con 0x7efd900ce3c0 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.257+0000 7efd95ca4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900712b0 0x7efd9015ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.257+0000 7efd954a3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd900cdfe0 0x7efd900c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.257+0000 7efd964a5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900ce3c0 0x7efd901f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.257+0000 7efd95ca4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900712b0 0x7efd9015ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39928/0 (socket says 172.21.15.133:39928) 2024-02-21T23:25:52.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.257+0000 7efd95ca4640 1 -- 172.21.15.133:0/2249653105 learned_addr learned my addr 172.21.15.133:0/2249653105 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:52.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.260+0000 7efd95ca4640 1 -- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900ce3c0 msgr2=0x7efd901f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.260+0000 7efd95ca4640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900ce3c0 0x7efd901f0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.260+0000 7efd95ca4640 1 -- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd900cdfe0 msgr2=0x7efd900c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.260+0000 7efd95ca4640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd900cdfe0 0x7efd900c6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.260+0000 7efd95ca4640 1 -- 172.21.15.133:0/2249653105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900c57a0 con 0x7efd900712b0 2024-02-21T23:25:52.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.262+0000 7efd95ca4640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900712b0 0x7efd9015ba00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c820 tx=0x7efd8c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:52.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.266+0000 7efd867fc640 1 -- 172.21.15.133:0/2249653105 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c015070 con 0x7efd900712b0 2024-02-21T23:25:52.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.266+0000 7efd867fc640 1 -- 172.21.15.133:0/2249653105 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c004240 con 0x7efd900712b0 2024-02-21T23:25:52.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.266+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd901f1270 con 0x7efd900712b0 2024-02-21T23:25:52.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.266+0000 7efd867fc640 1 -- 172.21.15.133:0/2249653105 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00ce10 con 0x7efd900712b0 2024-02-21T23:25:52.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.266+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd900c6230 con 0x7efd900712b0 2024-02-21T23:25:52.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.267+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9006a130 con 0x7efd900712b0 2024-02-21T23:25:52.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.510+0000 7efd867fc640 1 -- 172.21.15.133:0/2249653105 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd8c007830 con 0x7efd900712b0 2024-02-21T23:25:52.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.510+0000 7efd867fc640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd7406b4d0 0x7efd7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.510+0000 7efd954a3640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd7406b4d0 0x7efd7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.511+0000 7efd954a3640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd7406b4d0 0x7efd7406d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efd80036000 tx=0x7efd8003b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:52.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.523+0000 7efd867fc640 1 -- 172.21.15.133:0/2249653105 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efd8c056d60 con 0x7efd900712b0 2024-02-21T23:25:52.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.523+0000 7efd867fc640 1 -- 172.21.15.133:0/2249653105 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7efd900c57a0 con 0x7efd900712b0 2024-02-21T23:25:52.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.743+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efd901569a0 con 0x7efd900712b0 2024-02-21T23:25:52.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.877+0000 7efd867fc640 1 -- 172.21.15.133:0/2249653105 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171792 (secure 0 0 0) 0x7efd8c058cc0 con 0x7efd900712b0 2024-02-21T23:25:52.865 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:52.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd7406b4d0 msgr2=0x7efd7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd7406b4d0 0x7efd7406d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efd80036000 tx=0x7efd8003b040 comp rx=0 tx=0).stop 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900712b0 msgr2=0x7efd9015ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900712b0 0x7efd9015ba00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c820 tx=0x7efd8c00cce0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 shutdown_connections 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd7406b4d0 0x7efd7406d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd900ce3c0 0x7efd901f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd900cdfe0 0x7efd900c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 --2- 172.21.15.133:0/2249653105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd900712b0 0x7efd9015ba00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 >> 172.21.15.133:0/2249653105 conn(0x7efd9006aec0 msgr2=0x7efd900c2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 shutdown_connections 2024-02-21T23:25:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.880+0000 7efd97f08640 1 -- 172.21.15.133:0/2249653105 wait complete. 2024-02-21T23:25:52.877 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"flags":0,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3089949271},{"type":"v1","addr":"172.21.15.142:6801","nonce":3089949271}]},"active_addr":"172.21.15.142:6801/3089949271","active_change":"2024-02-21T23:24:54.456385+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":1012843906}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":2786143566}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":2446360723}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":1804669070}]}]} 2024-02-21T23:25:52.880 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-02-21T23:25:52.880 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-02-21T23:25:52.880 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.994+0000 7f2b6e0de640 1 Processor -- start 2024-02-21T23:25:52.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.994+0000 7f2b6e0de640 1 -- start start 2024-02-21T23:25:52.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.994+0000 7f2b6e0de640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2b68071f60 0x7f2b680c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.995+0000 7f2b6e0de640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b680c53e0 con 0x7f2b680c7ac0 2024-02-21T23:25:52.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.995+0000 7f2b6e0de640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b680c5680 con 0x7f2b68150a30 2024-02-21T23:25:52.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.995+0000 7f2b6e0de640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b68071860 con 0x7f2b68071f60 2024-02-21T23:25:52.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.995+0000 7f2b677fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f2b68150a30 0x7f2b68071150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:37320/0 (socket says 172.21.15.133:37320) 2024-02-21T23:25:52.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.995+0000 7f2b677fe640 1 -- 172.21.15.133:0/344008273 learned_addr learned my addr 172.21.15.133:0/344008273 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:52.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.995+0000 7f2b66ffd640 1 --2- 172.21.15.133:0/344008273 >> v2:172.21.15.133:3300/0 conn(0x7f2b68071f60 0x7f2b680c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.998+0000 7f2b667fc640 1 -- 172.21.15.133:0/344008273 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834545873 0 0) 0x7f2b680c5680 con 0x7f2b68150a30 2024-02-21T23:25:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:52.998+0000 7f2b667fc640 1 -- 172.21.15.133:0/344008273 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b50003520 con 0x7f2b68150a30 2024-02-21T23:25:52.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.000+0000 7f2b66ffd640 1 -- 172.21.15.133:0/344008273 >> v1:172.21.15.142:6789/0 conn(0x7f2b68150a30 legacy=0x7f2b68071150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.000+0000 7f2b66ffd640 1 -- 172.21.15.133:0/344008273 >> v1:172.21.15.133:6789/0 conn(0x7f2b680c7ac0 legacy=0x7f2b6806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.000+0000 7f2b66ffd640 1 -- 172.21.15.133:0/344008273 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680c5960 con 0x7f2b68071f60 2024-02-21T23:25:52.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.003+0000 7f2b66ffd640 1 --2- 172.21.15.133:0/344008273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b68071f60 0x7f2b680c2b40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2b580097a0 tx=0x7f2b5802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d434c56dd84bc5f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.007+0000 7f2b667fc640 1 -- 172.21.15.133:0/344008273 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5802f810 con 0x7f2b68071f60 2024-02-21T23:25:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.007+0000 7f2b667fc640 1 -- 172.21.15.133:0/344008273 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5802f9b0 con 0x7f2b68071f60 2024-02-21T23:25:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.007+0000 7f2b667fc640 1 -- 172.21.15.133:0/344008273 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5802fd80 con 0x7f2b68071f60 2024-02-21T23:25:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.008+0000 7f2b6e0de640 1 -- 172.21.15.133:0/344008273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b68071f60 msgr2=0x7f2b680c2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.008+0000 7f2b6e0de640 1 --2- 172.21.15.133:0/344008273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b68071f60 0x7f2b680c2b40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2b580097a0 tx=0x7f2b5802b9f0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.008+0000 7f2b6e0de640 1 -- 172.21.15.133:0/344008273 shutdown_connections 2024-02-21T23:25:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.008+0000 7f2b6e0de640 1 --2- 172.21.15.133:0/344008273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b68071f60 0x7f2b680c2b40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.008+0000 7f2b6e0de640 1 -- 172.21.15.133:0/344008273 >> 172.21.15.133:0/344008273 conn(0x7f2b6806aec0 msgr2=0x7f2b6806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.008+0000 7f2b6e0de640 1 -- 172.21.15.133:0/344008273 shutdown_connections 2024-02-21T23:25:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.008+0000 7f2b6e0de640 1 -- 172.21.15.133:0/344008273 wait complete. 2024-02-21T23:25:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.010+0000 7f2b6e0de640 1 Processor -- start 2024-02-21T23:25:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.010+0000 7f2b6e0de640 1 -- start start 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.010+0000 7f2b6e0de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b68071f60 0x7f2b68142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b6e0de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b680c7ac0 0x7f2b68142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b6e0de640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b68150a30 0x7f2b6813eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b6e0de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68072a70 con 0x7f2b680c7ac0 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b6e0de640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68072540 con 0x7f2b68150a30 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b6e0de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68150e10 con 0x7f2b68071f60 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b66ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b680c7ac0 0x7f2b68142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b67fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b68150a30 0x7f2b6813eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b68071f60 0x7f2b68142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b67fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b68150a30 0x7f2b6813eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42472/0 (socket says 172.21.15.133:42472) 2024-02-21T23:25:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.011+0000 7f2b67fff640 1 -- 172.21.15.133:0/2437507249 learned_addr learned my addr 172.21.15.133:0/2437507249 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.015+0000 7f2b66ffd640 1 -- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b68071f60 msgr2=0x7f2b68142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.015+0000 7f2b66ffd640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b68071f60 0x7f2b68142760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.015+0000 7f2b66ffd640 1 -- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b68150a30 msgr2=0x7f2b6813eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.015+0000 7f2b66ffd640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b68150a30 0x7f2b6813eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.015+0000 7f2b66ffd640 1 -- 172.21.15.133:0/2437507249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68137130 con 0x7f2b680c7ac0 2024-02-21T23:25:53.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.017+0000 7f2b66ffd640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b680c7ac0 0x7f2b68142ca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b58033000 tx=0x7f2b580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.021+0000 7f2b4ffff640 1 -- 172.21.15.133:0/2437507249 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b580376d0 con 0x7f2b680c7ac0 2024-02-21T23:25:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.021+0000 7f2b4ffff640 1 -- 172.21.15.133:0/2437507249 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b58037890 con 0x7f2b680c7ac0 2024-02-21T23:25:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.021+0000 7f2b4ffff640 1 -- 172.21.15.133:0/2437507249 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58037c80 con 0x7f2b680c7ac0 2024-02-21T23:25:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.021+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b680cb7c0 con 0x7f2b680c7ac0 2024-02-21T23:25:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.021+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b68138e40 con 0x7f2b680c7ac0 2024-02-21T23:25:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.022+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b680c7010 con 0x7f2b680c7ac0 2024-02-21T23:25:53.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.040+0000 7f2b4ffff640 1 -- 172.21.15.133:0/2437507249 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b58049020 con 0x7f2b680c7ac0 2024-02-21T23:25:53.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.041+0000 7f2b4ffff640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2b5406b4d0 0x7f2b5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:53.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.041+0000 7f2b677fe640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2b5406b4d0 0x7f2b5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:53.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.041+0000 7f2b677fe640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2b5406b4d0 0x7f2b5406d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005880 tx=0x7f2b5c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.278+0000 7f2b4ffff640 1 -- 172.21.15.133:0/2437507249 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2b58079660 con 0x7f2b680c7ac0 2024-02-21T23:25:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.278+0000 7f2b4ffff640 1 -- 172.21.15.133:0/2437507249 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f2b680c7010 con 0x7f2b680c7ac0 2024-02-21T23:25:53.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.451+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b6806a130 con 0x7f2b680c7ac0 2024-02-21T23:25:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.489+0000 7f2b4ffff640 1 -- 172.21.15.133:0/2437507249 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f2b58050020 con 0x7f2b680c7ac0 2024-02-21T23:25:53.475 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:53.476 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2b5406b4d0 msgr2=0x7f2b5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2b5406b4d0 0x7f2b5406d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005880 tx=0x7f2b5c009040 comp rx=0 tx=0).stop 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b680c7ac0 msgr2=0x7f2b68142ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b680c7ac0 0x7f2b68142ca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b58033000 tx=0x7f2b580046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 shutdown_connections 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2b5406b4d0 0x7f2b5406d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b68150a30 0x7f2b6813eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b680c7ac0 0x7f2b68142ca0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 --2- 172.21.15.133:0/2437507249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b68071f60 0x7f2b68142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.492+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 >> 172.21.15.133:0/2437507249 conn(0x7f2b6806aec0 msgr2=0x7f2b6806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.493+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 shutdown_connections 2024-02-21T23:25:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.493+0000 7f2b6e0de640 1 -- 172.21.15.133:0/2437507249 wait complete. 2024-02-21T23:25:53.489 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-02-21T23:25:53.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:53.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.604+0000 7fab55dcc640 1 Processor -- start 2024-02-21T23:25:53.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.604+0000 7fab55dcc640 1 -- start start 2024-02-21T23:25:53.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.604+0000 7fab55dcc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 0x7fab5006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:53.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.604+0000 7fab55dcc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5015a720 con 0x7fab5006ff00 2024-02-21T23:25:53.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.604+0000 7fab55dcc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5015a9e0 con 0x7fab5015a290 2024-02-21T23:25:53.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.604+0000 7fab55dcc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5006c2f0 con 0x7fab50154450 2024-02-21T23:25:53.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.605+0000 7fab4ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fab5006ff00 0x7fab500c3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51356/0 (socket says 172.21.15.133:51356) 2024-02-21T23:25:53.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.605+0000 7fab4ffff640 1 -- 172.21.15.133:0/1768156787 learned_addr learned my addr 172.21.15.133:0/1768156787 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:53.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.605+0000 7fab4f7fe640 1 --2- 172.21.15.133:0/1768156787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 0x7fab5006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:53.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.608+0000 7fab4e7fc640 1 -- 172.21.15.133:0/1768156787 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798264454 0 0) 0x7fab5015a720 con 0x7fab5006ff00 2024-02-21T23:25:53.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.608+0000 7fab4e7fc640 1 -- 172.21.15.133:0/1768156787 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab38003520 con 0x7fab5006ff00 2024-02-21T23:25:53.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.608+0000 7fab4f7fe640 1 -- 172.21.15.133:0/1768156787 >> v1:172.21.15.142:6789/0 conn(0x7fab5015a290 legacy=0x7fab500c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:53.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.608+0000 7fab4f7fe640 1 -- 172.21.15.133:0/1768156787 >> v1:172.21.15.133:6789/0 conn(0x7fab5006ff00 legacy=0x7fab500c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:53.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.608+0000 7fab4f7fe640 1 -- 172.21.15.133:0/1768156787 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5015acc0 con 0x7fab50154450 2024-02-21T23:25:53.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.610+0000 7fab4f7fe640 1 --2- 172.21.15.133:0/1768156787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 0x7fab5006f9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fab400097c0 tx=0x7fab4002b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d66aeab0b28bae server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.613+0000 7fab4e7fc640 1 -- 172.21.15.133:0/1768156787 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40034070 con 0x7fab50154450 2024-02-21T23:25:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.613+0000 7fab4e7fc640 1 -- 172.21.15.133:0/1768156787 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab40030880 con 0x7fab50154450 2024-02-21T23:25:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.613+0000 7fab4e7fc640 1 -- 172.21.15.133:0/1768156787 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40030c50 con 0x7fab50154450 2024-02-21T23:25:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.614+0000 7fab55dcc640 1 -- 172.21.15.133:0/1768156787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 msgr2=0x7fab5006f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.614+0000 7fab55dcc640 1 --2- 172.21.15.133:0/1768156787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 0x7fab5006f9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fab400097c0 tx=0x7fab4002b9c0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.614+0000 7fab55dcc640 1 -- 172.21.15.133:0/1768156787 shutdown_connections 2024-02-21T23:25:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.614+0000 7fab55dcc640 1 --2- 172.21.15.133:0/1768156787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 0x7fab5006f9c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.614+0000 7fab55dcc640 1 -- 172.21.15.133:0/1768156787 >> 172.21.15.133:0/1768156787 conn(0x7fab50070ec0 msgr2=0x7fab500712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.614+0000 7fab55dcc640 1 -- 172.21.15.133:0/1768156787 shutdown_connections 2024-02-21T23:25:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.614+0000 7fab55dcc640 1 -- 172.21.15.133:0/1768156787 wait complete. 2024-02-21T23:25:53.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.615+0000 7fab55dcc640 1 Processor -- start 2024-02-21T23:25:53.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.615+0000 7fab55dcc640 1 -- start start 2024-02-21T23:25:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.616+0000 7fab55dcc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab5006ff00 0x7fab50144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.616+0000 7fab55dcc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 0x7fab501450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.616+0000 7fab55dcc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab50151c70 0x7fab501525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.616+0000 7fab55dcc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab500c8940 con 0x7fab50151c70 2024-02-21T23:25:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.616+0000 7fab55dcc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5006f4a0 con 0x7fab5006ff00 2024-02-21T23:25:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.616+0000 7fab55dcc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab500c8ac0 con 0x7fab50154450 2024-02-21T23:25:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.616+0000 7fab4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab50151c70 0x7fab501525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.616+0000 7fab4effd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 0x7fab501450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.617+0000 7fab4f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab5006ff00 0x7fab50144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.617+0000 7fab4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab50151c70 0x7fab501525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39946/0 (socket says 172.21.15.133:39946) 2024-02-21T23:25:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.617+0000 7fab4ffff640 1 -- 172.21.15.133:0/951834515 learned_addr learned my addr 172.21.15.133:0/951834515 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.620+0000 7fab4ffff640 1 -- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 msgr2=0x7fab501450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.620+0000 7fab4ffff640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 0x7fab501450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.620+0000 7fab4ffff640 1 -- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab5006ff00 msgr2=0x7fab50144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.620+0000 7fab4ffff640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab5006ff00 0x7fab50144b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.620+0000 7fab4ffff640 1 -- 172.21.15.133:0/951834515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab501f09f0 con 0x7fab50151c70 2024-02-21T23:25:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.622+0000 7fab4ffff640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab50151c70 0x7fab501525e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab4800cb40 tx=0x7fab4800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.626+0000 7fab37fff640 1 -- 172.21.15.133:0/951834515 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48017070 con 0x7fab50151c70 2024-02-21T23:25:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.626+0000 7fab37fff640 1 -- 172.21.15.133:0/951834515 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4800aca0 con 0x7fab50151c70 2024-02-21T23:25:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.626+0000 7fab37fff640 1 -- 172.21.15.133:0/951834515 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48012620 con 0x7fab50151c70 2024-02-21T23:25:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.626+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab50153660 con 0x7fab50151c70 2024-02-21T23:25:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.626+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab50153c20 con 0x7fab50151c70 2024-02-21T23:25:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.627+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5006a130 con 0x7fab50151c70 2024-02-21T23:25:53.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.648+0000 7fab37fff640 1 -- 172.21.15.133:0/951834515 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab4801a070 con 0x7fab50151c70 2024-02-21T23:25:53.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.649+0000 7fab37fff640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab3c06b520 0x7fab3c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:53.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.649+0000 7fab4f7fe640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab3c06b520 0x7fab3c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:53.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.649+0000 7fab4f7fe640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab3c06b520 0x7fab3c06d9e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab40036000 tx=0x7fab4003b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:53.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.651+0000 7fab37fff640 1 -- 172.21.15.133:0/951834515 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fab48056d60 con 0x7fab50151c70 2024-02-21T23:25:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:53.888+0000 7fab37fff640 1 -- 172.21.15.133:0/951834515 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fab48021020 con 0x7fab50151c70 2024-02-21T23:25:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.068+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab50149cd0 con 0x7fab50151c70 2024-02-21T23:25:54.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.107+0000 7fab37fff640 1 -- 172.21.15.133:0/951834515 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fab48058cc0 con 0x7fab50151c70 2024-02-21T23:25:54.093 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:54.094 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:22.292692+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab3c06b520 msgr2=0x7fab3c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab3c06b520 0x7fab3c06d9e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab40036000 tx=0x7fab4003b040 comp rx=0 tx=0).stop 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab50151c70 msgr2=0x7fab501525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab50151c70 0x7fab501525e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab4800cb40 tx=0x7fab4800a060 comp rx=0 tx=0).stop 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 shutdown_connections 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab3c06b520 0x7fab3c06d9e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab50151c70 0x7fab501525e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab50154450 0x7fab501450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 --2- 172.21.15.133:0/951834515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab5006ff00 0x7fab50144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 >> 172.21.15.133:0/951834515 conn(0x7fab50070ec0 msgr2=0x7fab501509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 shutdown_connections 2024-02-21T23:25:54.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.110+0000 7fab55dcc640 1 -- 172.21.15.133:0/951834515 wait complete. 2024-02-21T23:25:54.107 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-02-21T23:25:54.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-02-21T23:25:54.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-02-21T23:25:54.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-02-21T23:25:54.109 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-02-21T23:25:54.109 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-02-21T23:25:54.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-02-21T23:25:54.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-02-21T23:25:54.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.296+0000 7f418aefc640 1 Processor -- start 2024-02-21T23:25:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.295+0000 7fcfb8f8e640 1 Processor -- start 2024-02-21T23:25:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.295+0000 7fcfb8f8e640 1 -- start start 2024-02-21T23:25:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.296+0000 7fcfb8f8e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcfb40c37e0 0x7fcfb40c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.296+0000 7fcfb8f8e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcfb4070ee0 0x7fcfb4071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7fcfb8f8e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb416a1a0 con 0x7fcfb40c7a20 2024-02-21T23:25:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7fcfb8f8e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb416a480 con 0x7fcfb40c37e0 2024-02-21T23:25:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7fcfb8f8e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb416a760 con 0x7fcfb4070ee0 2024-02-21T23:25:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.296+0000 7f418aefc640 1 -- start start 2024-02-21T23:25:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7fcfb37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcfb40c37e0 0x7fcfb40c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.298+0000 7f744f620640 1 Processor -- start 2024-02-21T23:25:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7f418aefc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 0x7f417c046460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7f418aefc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f417c04ba60 0x7f417c04be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7f418aefc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f417c0e3090 0x7f417c0e3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7f418aefc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c045450 con 0x7f417c04ba60 2024-02-21T23:25:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7f418aefc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c045fd0 con 0x7f417c0e3090 2024-02-21T23:25:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.297+0000 7f418aefc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0dd7a0 con 0x7f417c0dd3c0 2024-02-21T23:25:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.298+0000 7fcfb2ffd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fcfb40c7a20 0x7fcfb40c8150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:37332/0 (socket says 172.21.15.133:37332) 2024-02-21T23:25:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.298+0000 7fcfb2ffd640 1 -- 172.21.15.133:0/2993845792 learned_addr learned my addr 172.21.15.133:0/2993845792 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.298+0000 7fcfb3fff640 1 --2- 172.21.15.133:0/2993845792 >> v2:172.21.15.142:3300/0 conn(0x7fcfb4070ee0 0x7fcfb4071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.298+0000 7f744f620640 1 -- start start 2024-02-21T23:25:54.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.298+0000 7f4189efa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 0x7f417c046460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.299+0000 7f162401c640 1 Processor -- start 2024-02-21T23:25:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.299+0000 7f4189efa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 0x7f417c046460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45726/0 (socket says 172.21.15.133:45726) 2024-02-21T23:25:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.299+0000 7f4189efa640 1 -- 172.21.15.133:0/2213481180 learned_addr learned my addr 172.21.15.133:0/2213481180 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.299+0000 7f162401c640 1 -- start start 2024-02-21T23:25:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.299+0000 7fcd293d4640 1 Processor -- start 2024-02-21T23:25:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.299+0000 7f98dab84640 1 Processor -- start 2024-02-21T23:25:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.299+0000 7f744f620640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806e3b0 0x7f74480c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f744f620640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f744806e790 0x7f7448070e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f744f620640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f744806ede0 0x7f7448153790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.299+0000 7f98dab84640 1 -- start start 2024-02-21T23:25:54.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f744f620640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7448073240 con 0x7f744806e790 2024-02-21T23:25:54.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f744f620640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f744815ddf0 con 0x7f744806ede0 2024-02-21T23:25:54.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f744f620640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744815e320 con 0x7f744806e3b0 2024-02-21T23:25:54.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f418a6fb640 1 --2- 172.21.15.133:0/2213481180 >> v2:172.21.15.142:3300/0 conn(0x7f417c0e3090 0x7f417c0e3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f162401c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c6f50 0x7f161c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f162401c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f161c0c7c10 0x7f161c0c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f162401c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161c0c85e0 con 0x7f161c154600 2024-02-21T23:25:54.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f162401c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161c0c74a0 con 0x7f161c0c7c10 2024-02-21T23:25:54.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7f162401c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0c7780 con 0x7f161c0c6f50 2024-02-21T23:25:54.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.299+0000 7fcd293d4640 1 -- start start 2024-02-21T23:25:54.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7fcd293d4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 0x7fcd1c0e33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7fcd293d4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd1c04c160 0x7fcd1c04c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7fcd293d4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c04cb30 con 0x7fcd1c006f20 2024-02-21T23:25:54.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7fcd293d4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c04bcd0 con 0x7fcd1c04c160 2024-02-21T23:25:54.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.300+0000 7fcd293d4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c04b9a0 con 0x7fcd1c0e6910 2024-02-21T23:25:54.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.301+0000 7f98dab84640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f98cc0e3790 0x7f98cc04be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.301+0000 7f98dab84640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc046b60 con 0x7f98cc0dd3b0 2024-02-21T23:25:54.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.301+0000 7f98dab84640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc0dd790 con 0x7f98cc04ba50 2024-02-21T23:25:54.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.301+0000 7f98dab84640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc045440 con 0x7f98cc0e3790 2024-02-21T23:25:54.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.302+0000 7f4189efa640 1 -- 172.21.15.133:0/2213481180 >> v2:172.21.15.142:3300/0 conn(0x7f417c0e3090 msgr2=0x7f417c0e3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.302+0000 7f4189efa640 1 --2- 172.21.15.133:0/2213481180 >> v2:172.21.15.142:3300/0 conn(0x7f417c0e3090 0x7f417c0e3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.302+0000 7f4189efa640 1 -- 172.21.15.133:0/2213481180 >> v2:172.21.15.133:3300/0 conn(0x7f417c04ba60 msgr2=0x7f417c04be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.302+0000 7f4189efa640 1 --2- 172.21.15.133:0/2213481180 >> v2:172.21.15.133:3300/0 conn(0x7f417c04ba60 0x7f417c04be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.302+0000 7f4189efa640 1 -- 172.21.15.133:0/2213481180 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c0e3e90 con 0x7f417c0dd3c0 2024-02-21T23:25:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.303+0000 7fcfb37fe640 1 -- 172.21.15.133:0/2993845792 >> v2:172.21.15.142:3300/0 conn(0x7fcfb4070ee0 msgr2=0x7fcfb4071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.303+0000 7fcfb37fe640 1 --2- 172.21.15.133:0/2993845792 >> v2:172.21.15.142:3300/0 conn(0x7fcfb4070ee0 0x7fcfb4071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.303+0000 7fcfb37fe640 1 -- 172.21.15.133:0/2993845792 >> v1:172.21.15.142:6789/0 conn(0x7fcfb40c7a20 legacy=0x7fcfb40c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.303+0000 7fcfb37fe640 1 -- 172.21.15.133:0/2993845792 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb416aa40 con 0x7fcfb40c37e0 2024-02-21T23:25:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.303+0000 7f744d3bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806e3b0 0x7f74480c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.303+0000 7f744d3bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806e3b0 0x7f74480c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45732/0 (socket says 172.21.15.133:45732) 2024-02-21T23:25:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.303+0000 7f744d3bc640 1 -- 172.21.15.133:0/72741602 learned_addr learned my addr 172.21.15.133:0/72741602 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.304+0000 7f7473a77640 1 Processor -- start 2024-02-21T23:25:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.304+0000 7f7473a77640 1 -- start start 2024-02-21T23:25:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.304+0000 7f744cbbb640 1 --2- 172.21.15.133:0/72741602 >> v2:172.21.15.133:3300/0 conn(0x7f744806e790 0x7f7448070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.305+0000 7f4189efa640 1 --2- 172.21.15.133:0/2213481180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 0x7f417c046460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f417c033e70 tx=0x7f418002b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b601709159580b server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.305+0000 7f7473a77640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f746c154600 0x7f746c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.305+0000 7f7473a77640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f746c0c6f50 0x7f746c0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.305+0000 7f7473a77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f746c0cab80 0x7f746c0c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.305+0000 7f7473a77640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c0c74a0 con 0x7f746c0c6f50 2024-02-21T23:25:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.305+0000 7f7473a77640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c0c7780 con 0x7f746c154600 2024-02-21T23:25:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.305+0000 7f7473a77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c15ab40 con 0x7f746c0cab80 2024-02-21T23:25:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.306+0000 7fc4fe3bb640 1 Processor -- start 2024-02-21T23:25:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.306+0000 7fc4fe3bb640 1 -- start start 2024-02-21T23:25:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.307+0000 7fcfb37fe640 1 --2- 172.21.15.133:0/2993845792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb40c37e0 0x7fcfb40c52e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40c7ee0 tx=0x7fcfac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2047d266184882 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.307+0000 7fc4fe3bb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc4f8150a20 0x7fc4f80720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.307+0000 7fc4fe3bb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f8073530 con 0x7fc4f815a0d0 2024-02-21T23:25:54.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.307+0000 7fc4fe3bb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f80c7490 con 0x7fc4f815a510 2024-02-21T23:25:54.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.307+0000 7fc4fe3bb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f815a980 con 0x7fc4f8150a20 2024-02-21T23:25:54.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.308+0000 7f744d3bc640 1 -- 172.21.15.133:0/72741602 >> v2:172.21.15.142:3300/0 conn(0x7f744806ede0 msgr2=0x7f7448153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.308+0000 7f744d3bc640 1 --2- 172.21.15.133:0/72741602 >> v2:172.21.15.142:3300/0 conn(0x7f744806ede0 0x7f7448153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.308+0000 7f744d3bc640 1 -- 172.21.15.133:0/72741602 >> v2:172.21.15.133:3300/0 conn(0x7f744806e790 msgr2=0x7f7448070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.308+0000 7f744d3bc640 1 --2- 172.21.15.133:0/72741602 >> v2:172.21.15.133:3300/0 conn(0x7f744806e790 0x7f7448070e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.308+0000 7f744d3bc640 1 -- 172.21.15.133:0/72741602 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744815e040 con 0x7f744806e3b0 2024-02-21T23:25:54.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.309+0000 7f16225b9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f161c0c7c10 0x7f161c0c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.309+0000 7f16225b9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f161c0c7c10 0x7f161c0c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42510/0 (socket says 172.21.15.133:42510) 2024-02-21T23:25:54.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.309+0000 7f16225b9640 1 -- 172.21.15.133:0/3333405384 learned_addr learned my addr 172.21.15.133:0/3333405384 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.310+0000 7f98d9121640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f98cc0e3790 0x7f98cc04be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.311+0000 7f98d9121640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f98cc0e3790 0x7f98cc04be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42524/0 (socket says 172.21.15.133:42524) 2024-02-21T23:25:54.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.311+0000 7f98d9121640 1 -- 172.21.15.133:0/1571552117 learned_addr learned my addr 172.21.15.133:0/1571552117 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.312+0000 7f4188ef8640 1 -- 172.21.15.133:0/2213481180 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180035020 con 0x7f417c0dd3c0 2024-02-21T23:25:54.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.312+0000 7f4188ef8640 1 -- 172.21.15.133:0/2213481180 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4180031870 con 0x7f417c0dd3c0 2024-02-21T23:25:54.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.312+0000 7f4188ef8640 1 -- 172.21.15.133:0/2213481180 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180031c40 con 0x7f417c0dd3c0 2024-02-21T23:25:54.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.312+0000 7f16225b9640 1 -- 172.21.15.133:0/3333405384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c6f50 msgr2=0x7f161c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.312+0000 7f16225b9640 1 --2- 172.21.15.133:0/3333405384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c6f50 0x7f161c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.312+0000 7f16225b9640 1 -- 172.21.15.133:0/3333405384 >> v1:172.21.15.142:6789/0 conn(0x7f161c154600 legacy=0x7f161c0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.312+0000 7f16225b9640 1 -- 172.21.15.133:0/3333405384 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c15ab40 con 0x7f161c0c7c10 2024-02-21T23:25:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f418aefc640 1 -- 172.21.15.133:0/2213481180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 msgr2=0x7f417c046460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f418aefc640 1 --2- 172.21.15.133:0/2213481180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 0x7f417c046460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f417c033e70 tx=0x7f418002b910 comp rx=0 tx=0).stop 2024-02-21T23:25:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f418aefc640 1 -- 172.21.15.133:0/2213481180 shutdown_connections 2024-02-21T23:25:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f418aefc640 1 --2- 172.21.15.133:0/2213481180 >> v2:172.21.15.142:3300/0 conn(0x7f417c0e3090 0x7f417c0e3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f418aefc640 1 --2- 172.21.15.133:0/2213481180 >> v2:172.21.15.133:3300/0 conn(0x7f417c04ba60 0x7f417c04be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f418aefc640 1 --2- 172.21.15.133:0/2213481180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 0x7f417c046460 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f418aefc640 1 -- 172.21.15.133:0/2213481180 >> 172.21.15.133:0/2213481180 conn(0x7f417c050180 msgr2=0x7f417c050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7fcd23fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcd1c006f20 0x7fcd1c007300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51358/0 (socket says 172.21.15.133:51358) 2024-02-21T23:25:54.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7fcd23fff640 1 -- 172.21.15.133:0/2998699823 learned_addr learned my addr 172.21.15.133:0/2998699823 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:54.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f418aefc640 1 -- 172.21.15.133:0/2213481180 shutdown_connections 2024-02-21T23:25:54.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f744d3bc640 1 --2- 172.21.15.133:0/72741602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806e3b0 0x7f74480c6ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74480c65a0 tx=0x7f743802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1d3b2cf9bb491aa server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.313+0000 7f418aefc640 1 -- 172.21.15.133:0/2213481180 wait complete. 2024-02-21T23:25:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.315+0000 7f418aefc640 1 Processor -- start 2024-02-21T23:25:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.315+0000 7f418aefc640 1 -- start start 2024-02-21T23:25:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.315+0000 7f418aefc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f417c04ba60 0x7f417c0e2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.315+0000 7f418aefc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 0x7f417c179a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.315+0000 7f418aefc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f417c0e3090 0x7f417c179f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.315+0000 7f418aefc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c045c60 con 0x7f417c0e3090 2024-02-21T23:25:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.315+0000 7f418aefc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c045980 con 0x7f417c04ba60 2024-02-21T23:25:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.315+0000 7f418aefc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c005e90 con 0x7f417c0dd3c0 2024-02-21T23:25:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.316+0000 7f98d9121640 1 -- 172.21.15.133:0/1571552117 >> v1:172.21.15.142:6789/0 conn(0x7f98cc04ba50 legacy=0x7f98cc0e3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.316+0000 7f98d9121640 1 -- 172.21.15.133:0/1571552117 >> v1:172.21.15.133:6789/0 conn(0x7f98cc0dd3b0 legacy=0x7f98cc046450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.316+0000 7f98d9121640 1 -- 172.21.15.133:0/1571552117 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0e3bc0 con 0x7f98cc0e3790 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb27fc640 1 -- 172.21.15.133:0/2993845792 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac039070 con 0x7fcfb40c37e0 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb27fc640 1 -- 172.21.15.133:0/2993845792 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac02cc80 con 0x7fcfb40c37e0 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb27fc640 1 -- 172.21.15.133:0/2993845792 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac034660 con 0x7fcfb40c37e0 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/2993845792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb40c37e0 msgr2=0x7fcfb40c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb8f8e640 1 --2- 172.21.15.133:0/2993845792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb40c37e0 0x7fcfb40c52e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40c7ee0 tx=0x7fcfac02bed0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/2993845792 shutdown_connections 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb8f8e640 1 --2- 172.21.15.133:0/2993845792 >> v2:172.21.15.142:3300/0 conn(0x7fcfb4070ee0 0x7fcfb4071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb8f8e640 1 --2- 172.21.15.133:0/2993845792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb40c37e0 0x7fcfb40c52e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/2993845792 >> 172.21.15.133:0/2993845792 conn(0x7fcfb40cb5f0 msgr2=0x7fcfb40cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/2993845792 shutdown_connections 2024-02-21T23:25:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.317+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/2993845792 wait complete. 2024-02-21T23:25:54.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.318+0000 7f16225b9640 1 --2- 172.21.15.133:0/3333405384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f161c0c7c10 0x7f161c0c80a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f160c009870 tx=0x7f160c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1088224070e13ac1 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.319+0000 7fcfb8f8e640 1 Processor -- start 2024-02-21T23:25:54.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.319+0000 7fcfb8f8e640 1 -- start start 2024-02-21T23:25:54.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.319+0000 7fcd237fe640 1 --2- 172.21.15.133:0/2998699823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 0x7fcd1c0e33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.319+0000 7fcfb8f8e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb4070ee0 0x7fcfb4158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.319+0000 7fcfb8f8e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb40c37e0 0x7fcfb40ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.319+0000 7fcfb8f8e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcfb40c7a20 0x7fcfb40cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.319+0000 7fcfb8f8e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c8860 con 0x7fcfb4070ee0 2024-02-21T23:25:54.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.319+0000 7fcfb8f8e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c5820 con 0x7fcfb40c37e0 2024-02-21T23:25:54.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.319+0000 7fcfb8f8e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c89e0 con 0x7fcfb40c7a20 2024-02-21T23:25:54.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.320+0000 7f7471813640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f746c154600 0x7f746c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.320+0000 7f98d9121640 1 --2- 172.21.15.133:0/1571552117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc0e3790 0x7f98cc04be70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98d404f7b0 tx=0x7f98d40a6fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=936e3abb85ed4f78 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.321+0000 7f7471813640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f746c154600 0x7f746c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42536/0 (socket says 172.21.15.133:42536) 2024-02-21T23:25:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.321+0000 7f7471813640 1 -- 172.21.15.133:0/2409868453 learned_addr learned my addr 172.21.15.133:0/2409868453 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.321+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2998699823 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820192260 0 0) 0x7fcd1c04cb30 con 0x7fcd1c006f20 2024-02-21T23:25:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.321+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2998699823 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd14003520 con 0x7fcd1c006f20 2024-02-21T23:25:54.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.322+0000 7f7472014640 1 --2- 172.21.15.133:0/2409868453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f746c0cab80 0x7f746c0c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.323+0000 7fcd28bd3640 1 --2- 172.21.15.133:0/2998699823 >> v2:172.21.15.142:3300/0 conn(0x7fcd1c04c160 0x7fcd1c04c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.324+0000 7fcd237fe640 1 -- 172.21.15.133:0/2998699823 >> v2:172.21.15.142:3300/0 conn(0x7fcd1c04c160 msgr2=0x7fcd1c04c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.324+0000 7fcd237fe640 1 --2- 172.21.15.133:0/2998699823 >> v2:172.21.15.142:3300/0 conn(0x7fcd1c04c160 0x7fcd1c04c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.324+0000 7fcd237fe640 1 -- 172.21.15.133:0/2998699823 >> v1:172.21.15.133:6789/0 conn(0x7fcd1c006f20 legacy=0x7fcd1c007300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.324+0000 7fcd237fe640 1 -- 172.21.15.133:0/2998699823 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c007a10 con 0x7fcd1c0e6910 2024-02-21T23:25:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.324+0000 7fcd28bd3640 1 --2- 172.21.15.133:0/2998699823 >> v2:172.21.15.142:3300/0 conn(0x7fcd1c04c160 0x7fcd1c04c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-21T23:25:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.325+0000 7f743ffff640 1 -- 172.21.15.133:0/72741602 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438035020 con 0x7f744806e3b0 2024-02-21T23:25:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.325+0000 7f743ffff640 1 -- 172.21.15.133:0/72741602 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7438031890 con 0x7f744806e3b0 2024-02-21T23:25:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.325+0000 7f743ffff640 1 -- 172.21.15.133:0/72741602 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438031ca0 con 0x7f744806e3b0 2024-02-21T23:25:54.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.325+0000 7f418a6fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f417c0e3090 0x7f417c179f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.325+0000 7f41896f9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 0x7f417c179a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f744f620640 1 -- 172.21.15.133:0/72741602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806e3b0 msgr2=0x7f74480c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f744f620640 1 --2- 172.21.15.133:0/72741602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806e3b0 0x7f74480c6ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74480c65a0 tx=0x7f743802b930 comp rx=0 tx=0).stop 2024-02-21T23:25:54.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f744f620640 1 -- 172.21.15.133:0/72741602 shutdown_connections 2024-02-21T23:25:54.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f744f620640 1 --2- 172.21.15.133:0/72741602 >> v2:172.21.15.142:3300/0 conn(0x7f744806ede0 0x7f7448153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f744f620640 1 --2- 172.21.15.133:0/72741602 >> v2:172.21.15.133:3300/0 conn(0x7f744806e790 0x7f7448070e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f744f620640 1 --2- 172.21.15.133:0/72741602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806e3b0 0x7f74480c6ac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f744f620640 1 -- 172.21.15.133:0/72741602 >> 172.21.15.133:0/72741602 conn(0x7f74480cb250 msgr2=0x7f74480cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f744f620640 1 -- 172.21.15.133:0/72741602 shutdown_connections 2024-02-21T23:25:54.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f744f620640 1 -- 172.21.15.133:0/72741602 wait complete. 2024-02-21T23:25:54.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f7471813640 1 -- 172.21.15.133:0/2409868453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f746c0cab80 msgr2=0x7f746c0c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f7471813640 1 --2- 172.21.15.133:0/2409868453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f746c0cab80 0x7f746c0c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f7471813640 1 -- 172.21.15.133:0/2409868453 >> v2:172.21.15.133:3300/0 conn(0x7f746c0c6f50 msgr2=0x7f746c0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f7471813640 1 --2- 172.21.15.133:0/2409868453 >> v2:172.21.15.133:3300/0 conn(0x7f746c0c6f50 0x7f746c0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f7471813640 1 -- 172.21.15.133:0/2409868453 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c15ae00 con 0x7f746c154600 2024-02-21T23:25:54.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f418a6fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f417c0e3090 0x7f417c179f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39980/0 (socket says 172.21.15.133:39980) 2024-02-21T23:25:54.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.326+0000 7f418a6fb640 1 -- 172.21.15.133:0/1830649292 learned_addr learned my addr 172.21.15.133:0/1830649292 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.327+0000 7fcfb37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb4070ee0 0x7fcfb4158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.327+0000 7fcfb37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb4070ee0 0x7fcfb4158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39992/0 (socket says 172.21.15.133:39992) 2024-02-21T23:25:54.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.327+0000 7fcfb37fe640 1 -- 172.21.15.133:0/1621766316 learned_addr learned my addr 172.21.15.133:0/1621766316 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.327+0000 7f744f620640 1 Processor -- start 2024-02-21T23:25:54.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.327+0000 7f744f620640 1 -- start start 2024-02-21T23:25:54.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.327+0000 7fcd237fe640 1 --2- 172.21.15.133:0/2998699823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 0x7fcd1c0e33a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd100097a0 tx=0x7fcd1002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207307b1be30b264 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.328+0000 7f744f620640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f744806e3b0 0x7f744815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.328+0000 7f744f620640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f744806e790 0x7f74480c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.328+0000 7f744f620640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806ede0 0x7f7448149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.328+0000 7f744f620640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c5fe0 con 0x7f744806e3b0 2024-02-21T23:25:54.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.328+0000 7f744f620640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c6630 con 0x7f744806e790 2024-02-21T23:25:54.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.328+0000 7f744f620640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c62c0 con 0x7f744806ede0 2024-02-21T23:25:54.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.328+0000 7fc4f7fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc4f8150a20 0x7fc4f80720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.330+0000 7fc4f7fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc4f8150a20 0x7fc4f80720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42538/0 (socket says 172.21.15.133:42538) 2024-02-21T23:25:54.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.330+0000 7fc4f7fff640 1 -- 172.21.15.133:0/466862466 learned_addr learned my addr 172.21.15.133:0/466862466 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.331+0000 7fcfb3fff640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcfb40c7a20 0x7fcfb40cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.332+0000 7f418a6fb640 1 -- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 msgr2=0x7f417c179a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.332+0000 7f418a6fb640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0dd3c0 0x7f417c179a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.332+0000 7f418a6fb640 1 -- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f417c04ba60 msgr2=0x7f417c0e2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.332+0000 7f418a6fb640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f417c04ba60 0x7f417c0e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.332+0000 7f418a6fb640 1 -- 172.21.15.133:0/1830649292 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c17ccf0 con 0x7f417c0e3090 2024-02-21T23:25:54.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.333+0000 7f98d37fe640 1 -- 172.21.15.133:0/1571552117 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40b4070 con 0x7f98cc0e3790 2024-02-21T23:25:54.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.333+0000 7f98d37fe640 1 -- 172.21.15.133:0/1571552117 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d40a7e20 con 0x7f98cc0e3790 2024-02-21T23:25:54.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.333+0000 7f98d37fe640 1 -- 172.21.15.133:0/1571552117 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40af640 con 0x7f98cc0e3790 2024-02-21T23:25:54.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.334+0000 7f98dab84640 1 -- 172.21.15.133:0/1571552117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc0e3790 msgr2=0x7f98cc04be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.334+0000 7f98dab84640 1 --2- 172.21.15.133:0/1571552117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc0e3790 0x7f98cc04be70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98d404f7b0 tx=0x7f98d40a6fb0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.334+0000 7f98dab84640 1 -- 172.21.15.133:0/1571552117 shutdown_connections 2024-02-21T23:25:54.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.334+0000 7f98dab84640 1 --2- 172.21.15.133:0/1571552117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc0e3790 0x7f98cc04be70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.334+0000 7f98dab84640 1 -- 172.21.15.133:0/1571552117 >> 172.21.15.133:0/1571552117 conn(0x7f98cc050170 msgr2=0x7f98cc050580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.334+0000 7f98dab84640 1 -- 172.21.15.133:0/1571552117 shutdown_connections 2024-02-21T23:25:54.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.334+0000 7f98dab84640 1 -- 172.21.15.133:0/1571552117 wait complete. 2024-02-21T23:25:54.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.335+0000 7fc4f7fff640 1 -- 172.21.15.133:0/466862466 >> v1:172.21.15.142:6789/0 conn(0x7fc4f815a510 legacy=0x7fc4f8160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.335+0000 7fc4f7fff640 1 -- 172.21.15.133:0/466862466 >> v1:172.21.15.133:6789/0 conn(0x7fc4f815a0d0 legacy=0x7fc4f80c2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.335+0000 7fc4f7fff640 1 -- 172.21.15.133:0/466862466 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8072c90 con 0x7fc4f8150a20 2024-02-21T23:25:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.336+0000 7f98dab84640 1 Processor -- start 2024-02-21T23:25:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.336+0000 7f98dab84640 1 -- start start 2024-02-21T23:25:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7f7471813640 1 --2- 172.21.15.133:0/2409868453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 0x7f746c0cbc00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f746c06d270 tx=0x7f746802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a0d1633fa5ef58 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7fcfb37fe640 1 -- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcfb40c7a20 msgr2=0x7fcfb40cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7fcfb37fe640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcfb40c7a20 0x7fcfb40cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7fcfb37fe640 1 -- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb40c37e0 msgr2=0x7fcfb40ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7fcfb37fe640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb40c37e0 0x7fcfb40ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7fcfb37fe640 1 -- 172.21.15.133:0/1621766316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb408ae90 con 0x7fcfb4070ee0 2024-02-21T23:25:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7f98dab84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98cc04ba50 0x7f98cc0d4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7f98dab84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98cc0dd3b0 0x7f98cc0d3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7f98dab84640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc0e3790 0x7f98cc04f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7f98dab84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc045690 con 0x7f98cc0dd3b0 2024-02-21T23:25:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7f98dab84640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc045fc0 con 0x7f98cc0e3790 2024-02-21T23:25:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.337+0000 7f98dab84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc045970 con 0x7f98cc04ba50 2024-02-21T23:25:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.339+0000 7fc4f7fff640 1 --2- 172.21.15.133:0/466862466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8150a20 0x7fc4f80720c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009870 tx=0x7fc4e802cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f6b572202a14d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.340+0000 7f744dbbd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806ede0 0x7f7448149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.341+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2998699823 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10034070 con 0x7fcd1c0e6910 2024-02-21T23:25:54.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.341+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2998699823 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10030610 con 0x7fcd1c0e6910 2024-02-21T23:25:54.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.341+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2998699823 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100309e0 con 0x7fcd1c0e6910 2024-02-21T23:25:54.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.341+0000 7f744dbbd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806ede0 0x7f7448149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45802/0 (socket says 172.21.15.133:45802) 2024-02-21T23:25:54.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.341+0000 7f744dbbd640 1 -- 172.21.15.133:0/177298284 learned_addr learned my addr 172.21.15.133:0/177298284 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7f418a6fb640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f417c0e3090 0x7f417c179f50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4184064a20 tx=0x7f4184068390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7fcd293d4640 1 -- 172.21.15.133:0/2998699823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 msgr2=0x7fcd1c0e33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7fcd293d4640 1 --2- 172.21.15.133:0/2998699823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 0x7fcd1c0e33a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd100097a0 tx=0x7fcd1002b960 comp rx=0 tx=0).stop 2024-02-21T23:25:54.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7fcd293d4640 1 -- 172.21.15.133:0/2998699823 shutdown_connections 2024-02-21T23:25:54.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7fcd293d4640 1 --2- 172.21.15.133:0/2998699823 >> v2:172.21.15.142:3300/0 conn(0x7fcd1c04c160 0x7fcd1c04c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7fcd293d4640 1 --2- 172.21.15.133:0/2998699823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 0x7fcd1c0e33a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7fcd293d4640 1 -- 172.21.15.133:0/2998699823 >> 172.21.15.133:0/2998699823 conn(0x7fcd1c006190 msgr2=0x7fcd1c0065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7fcd293d4640 1 -- 172.21.15.133:0/2998699823 shutdown_connections 2024-02-21T23:25:54.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7fcd293d4640 1 -- 172.21.15.133:0/2998699823 wait complete. 2024-02-21T23:25:54.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.342+0000 7fcfb37fe640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb4070ee0 0x7fcfb4158bd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009e80 tx=0x7fcfac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.343+0000 7fcd293d4640 1 Processor -- start 2024-02-21T23:25:54.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.343+0000 7fcd293d4640 1 -- start start 2024-02-21T23:25:54.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.344+0000 7fcd293d4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1c006f20 0x7fcd1c179850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.344+0000 7fcd293d4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1c04c160 0x7fcd1c04fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.344+0000 7fcd293d4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 0x7fcd1c16d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.344+0000 7fcd293d4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0e3fa0 con 0x7fcd1c04c160 2024-02-21T23:25:54.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.344+0000 7fcd293d4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0e3cc0 con 0x7fcd1c006f20 2024-02-21T23:25:54.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.344+0000 7fcd293d4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0e4280 con 0x7fcd1c0e6910 2024-02-21T23:25:54.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.346+0000 7f744dbbd640 1 -- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f744806e790 msgr2=0x7f74480c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.346+0000 7f744dbbd640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f744806e790 0x7f74480c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.346+0000 7f744dbbd640 1 -- 172.21.15.133:0/177298284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f744806e3b0 msgr2=0x7f744815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.346+0000 7f744dbbd640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f744806e3b0 0x7f744815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.346+0000 7f744dbbd640 1 -- 172.21.15.133:0/177298284 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74480c55b0 con 0x7f744806ede0 2024-02-21T23:25:54.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.347+0000 7f1620db6640 1 -- 172.21.15.133:0/3333405384 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c035020 con 0x7f161c0c7c10 2024-02-21T23:25:54.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.347+0000 7f1620db6640 1 -- 172.21.15.133:0/3333405384 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c031640 con 0x7f161c0c7c10 2024-02-21T23:25:54.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.347+0000 7f1620db6640 1 -- 172.21.15.133:0/3333405384 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c02d040 con 0x7f161c0c7c10 2024-02-21T23:25:54.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.347+0000 7fcf9bfff640 1 -- 172.21.15.133:0/1621766316 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac039050 con 0x7fcfb4070ee0 2024-02-21T23:25:54.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.347+0000 7fcf9bfff640 1 -- 172.21.15.133:0/1621766316 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac045070 con 0x7fcfb4070ee0 2024-02-21T23:25:54.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.347+0000 7fcf9bfff640 1 -- 172.21.15.133:0/1621766316 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac03e850 con 0x7fcfb4070ee0 2024-02-21T23:25:54.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.347+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb40baab0 con 0x7fcfb4070ee0 2024-02-21T23:25:54.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb40bb070 con 0x7fcfb4070ee0 2024-02-21T23:25:54.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f162401c640 1 -- 172.21.15.133:0/3333405384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f161c0c7c10 msgr2=0x7f161c0c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f162401c640 1 --2- 172.21.15.133:0/3333405384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f161c0c7c10 0x7f161c0c80a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f160c009870 tx=0x7f160c02b8f0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f162401c640 1 -- 172.21.15.133:0/3333405384 shutdown_connections 2024-02-21T23:25:54.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f162401c640 1 --2- 172.21.15.133:0/3333405384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c6f50 0x7f161c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f162401c640 1 --2- 172.21.15.133:0/3333405384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f161c0c7c10 0x7f161c0c80a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f162401c640 1 -- 172.21.15.133:0/3333405384 >> 172.21.15.133:0/3333405384 conn(0x7f161c062000 msgr2=0x7f161c0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f162401c640 1 -- 172.21.15.133:0/3333405384 shutdown_connections 2024-02-21T23:25:54.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f7470811640 1 -- 172.21.15.133:0/2409868453 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468035020 con 0x7f746c154600 2024-02-21T23:25:54.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcfb40ba860 con 0x7fcfb4070ee0 2024-02-21T23:25:54.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f7470811640 1 -- 172.21.15.133:0/2409868453 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7468031870 con 0x7f746c154600 2024-02-21T23:25:54.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.348+0000 7f162401c640 1 -- 172.21.15.133:0/3333405384 wait complete. 2024-02-21T23:25:54.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.349+0000 7f7470811640 1 -- 172.21.15.133:0/2409868453 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468031c40 con 0x7f746c154600 2024-02-21T23:25:54.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.349+0000 7f162401c640 1 Processor -- start 2024-02-21T23:25:54.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.349+0000 7f162401c640 1 -- start start 2024-02-21T23:25:54.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.349+0000 7f7473a77640 1 -- 172.21.15.133:0/2409868453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 msgr2=0x7f746c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.349+0000 7f7473a77640 1 --2- 172.21.15.133:0/2409868453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 0x7f746c0cbc00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f746c06d270 tx=0x7f746802b910 comp rx=0 tx=0).stop 2024-02-21T23:25:54.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f7473a77640 1 -- 172.21.15.133:0/2409868453 shutdown_connections 2024-02-21T23:25:54.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f7473a77640 1 --2- 172.21.15.133:0/2409868453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f746c0cab80 0x7f746c0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f7473a77640 1 --2- 172.21.15.133:0/2409868453 >> v2:172.21.15.133:3300/0 conn(0x7f746c0c6f50 0x7f746c0ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f7473a77640 1 --2- 172.21.15.133:0/2409868453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 0x7f746c0cbc00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f7473a77640 1 -- 172.21.15.133:0/2409868453 >> 172.21.15.133:0/2409868453 conn(0x7f746c062000 msgr2=0x7f746c0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f7473a77640 1 -- 172.21.15.133:0/2409868453 shutdown_connections 2024-02-21T23:25:54.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f7473a77640 1 -- 172.21.15.133:0/2409868453 wait complete. 2024-02-21T23:25:54.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f162401c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f161c0c6f50 0x7f161c15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f162401c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c7c10 0x7f161c159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f162401c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f161c154600 0x7f161c0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f162401c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0cb860 con 0x7f161c154600 2024-02-21T23:25:54.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f162401c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0cb580 con 0x7f161c0c6f50 2024-02-21T23:25:54.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f162401c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0b5720 con 0x7f161c0c7c10 2024-02-21T23:25:54.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.350+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98cc0dd3b0 0x7f98cc0d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.351+0000 7f744dbbd640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806ede0 0x7f7448149380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f744000b9a0 tx=0x7f744000be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.351+0000 7f7473a77640 1 Processor -- start 2024-02-21T23:25:54.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.351+0000 7f7473a77640 1 -- start start 2024-02-21T23:25:54.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.351+0000 7f16215b7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c7c10 0x7f161c159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.352+0000 7fcd28bd3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 0x7fcd1c16d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.352+0000 7f7473a77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f746c0c6f50 0x7f746c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.352+0000 7f7473a77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f746c0cab80 0x7f746c147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.352+0000 7f7473a77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 0x7f746c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.352+0000 7f7473a77640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0b5720 con 0x7f746c0cab80 2024-02-21T23:25:54.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.352+0000 7f7473a77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0cb860 con 0x7f746c154600 2024-02-21T23:25:54.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.352+0000 7f7473a77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0b58a0 con 0x7f746c0c6f50 2024-02-21T23:25:54.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.352+0000 7fcd237fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1c04c160 0x7fcd1c04fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7f417a7fc640 1 -- 172.21.15.133:0/1830649292 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184094670 con 0x7f417c0e3090 2024-02-21T23:25:54.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7f417a7fc640 1 -- 172.21.15.133:0/1830649292 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418408f070 con 0x7f417c0e3090 2024-02-21T23:25:54.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7f417a7fc640 1 -- 172.21.15.133:0/1830649292 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184094a10 con 0x7f417c0e3090 2024-02-21T23:25:54.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c17cfd0 con 0x7f417c0e3090 2024-02-21T23:25:54.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c17d590 con 0x7f417c0e3090 2024-02-21T23:25:54.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98cc0dd3b0 0x7f98cc0d3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40008/0 (socket says 172.21.15.133:40008) 2024-02-21T23:25:54.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7f98d3fff640 1 -- 172.21.15.133:0/2530679820 learned_addr learned my addr 172.21.15.133:0/2530679820 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7fcd237fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1c04c160 0x7fcd1c04fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40024/0 (socket says 172.21.15.133:40024) 2024-02-21T23:25:54.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7fcd237fe640 1 -- 172.21.15.133:0/3861157775 learned_addr learned my addr 172.21.15.133:0/3861157775 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7fcd23fff640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1c006f20 0x7fcd1c179850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.353+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f417c005390 con 0x7f417c0e3090 2024-02-21T23:25:54.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.354+0000 7f7472014640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 0x7f746c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.355+0000 7f16215b7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c7c10 0x7f161c159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45826/0 (socket says 172.21.15.133:45826) 2024-02-21T23:25:54.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.355+0000 7f16215b7640 1 -- 172.21.15.133:0/202493326 learned_addr learned my addr 172.21.15.133:0/202493326 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.355+0000 7f7472014640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 0x7f746c0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42614/0 (socket says 172.21.15.133:42614) 2024-02-21T23:25:54.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.355+0000 7f7472014640 1 -- 172.21.15.133:0/688607958 learned_addr learned my addr 172.21.15.133:0/688607958 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.355+0000 7f98d8920640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98cc04ba50 0x7f98cc0d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.358+0000 7fc4f6ffd640 1 -- 172.21.15.133:0/466862466 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e803a070 con 0x7fc4f8150a20 2024-02-21T23:25:54.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.358+0000 7f1621db8640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f161c0c6f50 0x7f161c15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.358+0000 7fc4f6ffd640 1 -- 172.21.15.133:0/466862466 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e802d9f0 con 0x7fc4f8150a20 2024-02-21T23:25:54.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.358+0000 7fc4f6ffd640 1 -- 172.21.15.133:0/466862466 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e802ddc0 con 0x7fc4f8150a20 2024-02-21T23:25:54.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.358+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/466862466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8150a20 msgr2=0x7fc4f80720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.358+0000 7fc4fe3bb640 1 --2- 172.21.15.133:0/466862466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8150a20 0x7fc4f80720c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009870 tx=0x7fc4e802cc40 comp rx=0 tx=0).stop 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.358+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/466862466 shutdown_connections 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.359+0000 7fc4fe3bb640 1 --2- 172.21.15.133:0/466862466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8150a20 0x7fc4f80720c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.359+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/466862466 >> 172.21.15.133:0/466862466 conn(0x7fc4f806f6e0 msgr2=0x7fc4f806faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.359+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/466862466 shutdown_connections 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.359+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/466862466 wait complete. 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.360+0000 7fcd28bd3640 1 -- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1c006f20 msgr2=0x7fcd1c179850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.360+0000 7fcd28bd3640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1c006f20 0x7fcd1c179850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.360+0000 7fcd28bd3640 1 -- 172.21.15.133:0/3861157775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1c04c160 msgr2=0x7fcd1c04fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.360+0000 7fcd28bd3640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1c04c160 0x7fcd1c04fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.360+0000 7fcd28bd3640 1 -- 172.21.15.133:0/3861157775 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0deae0 con 0x7fcd1c0e6910 2024-02-21T23:25:54.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.360+0000 7fc4fe3bb640 1 Processor -- start 2024-02-21T23:25:54.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.360+0000 7fc4fe3bb640 1 -- start start 2024-02-21T23:25:54.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7f7472014640 1 -- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f746c0c6f50 msgr2=0x7f746c159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7f7472014640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f746c0c6f50 0x7f746c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7f7472014640 1 -- 172.21.15.133:0/688607958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f746c0cab80 msgr2=0x7f746c147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7f7472014640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f746c0cab80 0x7f746c147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7f7472014640 1 -- 172.21.15.133:0/688607958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c1f34c0 con 0x7f746c154600 2024-02-21T23:25:54.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7fc4fe3bb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4f815a0d0 0x7fc4f81591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7fc4fe3bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f815a510 0x7fc4f81457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7fc4fe3bb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8154750 0x7fc4f8145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7fc4fe3bb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8159c40 con 0x7fc4f815a510 2024-02-21T23:25:54.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7fc4fe3bb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80c7240 con 0x7fc4f8154750 2024-02-21T23:25:54.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.361+0000 7fc4fe3bb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f81564a0 con 0x7fc4f815a0d0 2024-02-21T23:25:54.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.364+0000 7f16215b7640 1 -- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f161c0c6f50 msgr2=0x7f161c15bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.364+0000 7f16215b7640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f161c0c6f50 0x7f161c15bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.364+0000 7f16215b7640 1 -- 172.21.15.133:0/202493326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f161c154600 msgr2=0x7f161c0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.364+0000 7f16215b7640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f161c154600 0x7f161c0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.364+0000 7f16215b7640 1 -- 172.21.15.133:0/202493326 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c1f78e0 con 0x7f161c0c7c10 2024-02-21T23:25:54.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.365+0000 7f7472014640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 0x7f746c0cdd20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7464007e40 tx=0x7f746400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.368+0000 7f98d9121640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc0e3790 0x7f98cc04f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.369+0000 7fcd28bd3640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 0x7fcd1c16d120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd24064a20 tx=0x7fcd24068aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.369+0000 7f743dffb640 1 -- 172.21.15.133:0/177298284 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440019070 con 0x7f744806ede0 2024-02-21T23:25:54.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.370+0000 7f743dffb640 1 -- 172.21.15.133:0/177298284 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744000cbc0 con 0x7f744806ede0 2024-02-21T23:25:54.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.370+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744815b120 con 0x7f744806ede0 2024-02-21T23:25:54.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.370+0000 7f743dffb640 1 -- 172.21.15.133:0/177298284 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440014440 con 0x7f744806ede0 2024-02-21T23:25:54.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.370+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74480c4ad0 con 0x7f744806ede0 2024-02-21T23:25:54.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.371+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f744815d260 con 0x7f744806ede0 2024-02-21T23:25:54.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.371+0000 7f16215b7640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c7c10 0x7f161c159320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f161800b440 tx=0x7f161800b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.374+0000 7f98d8920640 1 -- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc0e3790 msgr2=0x7f98cc04f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.374+0000 7f98d8920640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc0e3790 0x7f98cc04f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.374+0000 7f98d8920640 1 -- 172.21.15.133:0/2530679820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98cc0dd3b0 msgr2=0x7f98cc0d3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.374+0000 7f98d8920640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98cc0dd3b0 0x7f98cc0d3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.374+0000 7f98d8920640 1 -- 172.21.15.133:0/2530679820 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0d4dd0 con 0x7f98cc04ba50 2024-02-21T23:25:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.376+0000 7f417a7fc640 1 -- 172.21.15.133:0/1830649292 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f418409e3f0 con 0x7f417c0e3090 2024-02-21T23:25:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.377+0000 7f74627fc640 1 -- 172.21.15.133:0/688607958 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464014620 con 0x7f746c154600 2024-02-21T23:25:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.377+0000 7f74627fc640 1 -- 172.21.15.133:0/688607958 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7464019070 con 0x7f746c154600 2024-02-21T23:25:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.377+0000 7f417a7fc640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f416806b4d0 0x7f416806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.377+0000 7f74627fc640 1 -- 172.21.15.133:0/688607958 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464014a20 con 0x7f746c154600 2024-02-21T23:25:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.377+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c1f37a0 con 0x7f746c154600 2024-02-21T23:25:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.377+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c1f3d60 con 0x7f746c154600 2024-02-21T23:25:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.377+0000 7f4189efa640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f416806b4d0 0x7f416806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.377+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f746c1f3270 con 0x7f746c154600 2024-02-21T23:25:54.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.378+0000 7fc4fc958640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8154750 0x7fc4f8145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.378+0000 7f4189efa640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f416806b4d0 0x7f416806d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4180036000 tx=0x7f4180030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.380+0000 7f98d8920640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98cc04ba50 0x7f98cc0d4890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c390 tx=0x7f98c800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.383+0000 7f16127fc640 1 -- 172.21.15.133:0/202493326 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618015020 con 0x7f161c0c7c10 2024-02-21T23:25:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.383+0000 7f16127fc640 1 -- 172.21.15.133:0/202493326 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1618011890 con 0x7f161c0c7c10 2024-02-21T23:25:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.383+0000 7f16127fc640 1 -- 172.21.15.133:0/202493326 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618011c80 con 0x7f161c0c7c10 2024-02-21T23:25:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.383+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c1f7bc0 con 0x7f161c0c7c10 2024-02-21T23:25:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.383+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c1f8180 con 0x7f161c0c7c10 2024-02-21T23:25:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.383+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f161c1f4260 con 0x7f161c0c7c10 2024-02-21T23:25:54.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.384+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4f815a0d0 0x7fc4f81591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.385+0000 7f417a7fc640 1 -- 172.21.15.133:0/1830649292 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f41840d2300 con 0x7f417c0e3090 2024-02-21T23:25:54.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.385+0000 7f417a7fc640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7f4168074b40 0x7f4168076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.385+0000 7f417a7fc640 1 -- 172.21.15.133:0/1830649292 --> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f418409ee50 con 0x7f4168074b40 2024-02-21T23:25:54.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.385+0000 7f417a7fc640 1 -- 172.21.15.133:0/1830649292 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41840d0500 con 0x7f417c0e3090 2024-02-21T23:25:54.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.386+0000 7f41896f9640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7f4168074b40 0x7f4168076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.386+0000 7fc4fc958640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8154750 0x7fc4f8145ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42626/0 (socket says 172.21.15.133:42626) 2024-02-21T23:25:54.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.386+0000 7fc4fc958640 1 -- 172.21.15.133:0/3291839298 learned_addr learned my addr 172.21.15.133:0/3291839298 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.386+0000 7f41896f9640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7f4168074b40 0x7f4168076f80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.387+0000 7f417a7fc640 1 -- 172.21.15.133:0/1830649292 <== osd.7 v2:172.21.15.142:6810/3083092904 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f418409ee50 con 0x7f4168074b40 2024-02-21T23:25:54.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.388+0000 7fcd20ff9640 1 -- 172.21.15.133:0/3861157775 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd240716a0 con 0x7fcd1c0e6910 2024-02-21T23:25:54.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.388+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c0dedc0 con 0x7fcd1c0e6910 2024-02-21T23:25:54.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.388+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c179050 con 0x7fcd1c0e6910 2024-02-21T23:25:54.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.388+0000 7fcd20ff9640 1 -- 172.21.15.133:0/3861157775 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd24071840 con 0x7fcd1c0e6910 2024-02-21T23:25:54.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.388+0000 7fcd20ff9640 1 -- 172.21.15.133:0/3861157775 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24071bf0 con 0x7fcd1c0e6910 2024-02-21T23:25:54.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.389+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd04001030 con 0x7fcd1c0e6910 2024-02-21T23:25:54.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.399+0000 7f74627fc640 1 -- 172.21.15.133:0/688607958 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7464014bc0 con 0x7f746c154600 2024-02-21T23:25:54.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.399+0000 7f98d17fa640 1 -- 172.21.15.133:0/2530679820 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8017070 con 0x7f98cc04ba50 2024-02-21T23:25:54.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.399+0000 7f98d17fa640 1 -- 172.21.15.133:0/2530679820 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c80052d0 con 0x7f98cc04ba50 2024-02-21T23:25:54.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.399+0000 7f98d17fa640 1 -- 172.21.15.133:0/2530679820 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8003830 con 0x7f98cc04ba50 2024-02-21T23:25:54.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.399+0000 7f74627fc640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f745006b520 0x7f745006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.399+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc178dd0 con 0x7f98cc04ba50 2024-02-21T23:25:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.400+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc0d5690 con 0x7f98cc04ba50 2024-02-21T23:25:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.400+0000 7f7471813640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f745006b520 0x7f745006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.400+0000 7f7471813640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f745006b520 0x7f745006d9e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7468036000 tx=0x7f7468030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.400+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98cc005380 con 0x7f98cc04ba50 2024-02-21T23:25:54.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.401+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 --> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f417c005390 con 0x7f4168074b40 2024-02-21T23:25:54.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f417a7fc640 1 -- 172.21.15.133:0/1830649292 <== osd.7 v2:172.21.15.142:6810/3083092904 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f417c005390 con 0x7f4168074b40 2024-02-21T23:25:54.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7f4168074b40 msgr2=0x7f4168076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7f4168074b40 0x7f4168076f80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f416806b4d0 msgr2=0x7f416806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f416806b4d0 0x7f416806d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4180036000 tx=0x7f4180030000 comp rx=0 tx=0).stop 2024-02-21T23:25:54.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f417c0e3090 msgr2=0x7f417c179f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 --2- 172.21.15.133:0/1830649292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f417c0e3090 0x7f417c179f50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4184064a20 tx=0x7f4184068390 comp rx=0 tx=0).stop 2024-02-21T23:25:54.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418a6fb640 1 -- 172.21.15.133:0/1830649292 reap_dead start 2024-02-21T23:25:54.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 shutdown_connections 2024-02-21T23:25:54.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 >> 172.21.15.133:0/1830649292 conn(0x7f417c050180 msgr2=0x7f417c0d8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 shutdown_connections 2024-02-21T23:25:54.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.402+0000 7f418aefc640 1 -- 172.21.15.133:0/1830649292 wait complete. 2024-02-21T23:25:54.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.403+0000 7fc4fc958640 1 -- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4f815a0d0 msgr2=0x7fc4f81591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.403+0000 7fc4fc958640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4f815a0d0 0x7fc4f81591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.403+0000 7fc4fc958640 1 -- 172.21.15.133:0/3291839298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f815a510 msgr2=0x7fc4f81457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.403+0000 7fc4fc958640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f815a510 0x7fc4f81457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.403+0000 7fc4fc958640 1 -- 172.21.15.133:0/3291839298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f81488e0 con 0x7fc4f8154750 2024-02-21T23:25:54.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.409+0000 7f74627fc640 1 -- 172.21.15.133:0/688607958 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7464057da0 con 0x7f746c154600 2024-02-21T23:25:54.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.409+0000 7f74627fc640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f7450074ba0 0x7f7450076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.409+0000 7f74627fc640 1 -- 172.21.15.133:0/688607958 --> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7464016e70 con 0x7f7450074ba0 2024-02-21T23:25:54.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.410+0000 7f7471012640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f7450074ba0 0x7f7450076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.410+0000 7f7471012640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f7450074ba0 0x7f7450076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.410+0000 7f74627fc640 1 -- 172.21.15.133:0/688607958 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7464055fa0 con 0x7f746c154600 2024-02-21T23:25:54.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.411+0000 7f74627fc640 1 -- 172.21.15.133:0/688607958 <== osd.0 v2:172.21.15.133:6800/2680579490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7464016e70 con 0x7f7450074ba0 2024-02-21T23:25:54.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.413+0000 7fc4fc958640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8154750 0x7fc4f8145ce0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000df40 tx=0x7fc4f000df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.399 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:54.400 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-02-21T23:25:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.413+0000 7fcf9bfff640 1 -- 172.21.15.133:0/1621766316 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfac03e9f0 con 0x7fcfb4070ee0 2024-02-21T23:25:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.413+0000 7fcf9bfff640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcfa006b500 0x7fcfa006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.414+0000 7fcfb2ffd640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcfa006b500 0x7fcfa006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.414+0000 7fcfb2ffd640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcfa006b500 0x7fcfa006d9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40c6660 tx=0x7fcfa4007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.414+0000 7fc4f4ff9640 1 -- 172.21.15.133:0/3291839298 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000e980 con 0x7fc4f8154750 2024-02-21T23:25:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.414+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f8148bc0 con 0x7fc4f8154750 2024-02-21T23:25:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.414+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f8149180 con 0x7fc4f8154750 2024-02-21T23:25:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.415+0000 7f743dffb640 1 -- 172.21.15.133:0/177298284 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f744001c070 con 0x7f744806ede0 2024-02-21T23:25:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.415+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4f806a130 con 0x7fc4f8154750 2024-02-21T23:25:54.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.416+0000 7f743dffb640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f742c06b520 0x7f742c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.416+0000 7fc4f4ff9640 1 -- 172.21.15.133:0/3291839298 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f000eb20 con 0x7fc4f8154750 2024-02-21T23:25:54.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.416+0000 7fc4f4ff9640 1 -- 172.21.15.133:0/3291839298 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0024060 con 0x7fc4f8154750 2024-02-21T23:25:54.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.416+0000 7f744d3bc640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f742c06b520 0x7f742c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.416+0000 7f744d3bc640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f742c06b520 0x7f742c06d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7438036000 tx=0x7f7438030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.417+0000 7fcf9bfff640 1 -- 172.21.15.133:0/1621766316 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcfac07a3c0 con 0x7fcfb4070ee0 2024-02-21T23:25:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.417+0000 7fcf9bfff640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fcfa0075770 0x7fcfa0077c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.417+0000 7fcf9bfff640 1 -- 172.21.15.133:0/1621766316 --> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcfa00782e0 con 0x7fcfa0075770 2024-02-21T23:25:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.417+0000 7fcf9bfff640 1 -- 172.21.15.133:0/1621766316 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfac078d50 con 0x7fcfb4070ee0 2024-02-21T23:25:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.418+0000 7fcfb3fff640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fcfa0075770 0x7fcfa0077c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.418+0000 7fcfb3fff640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fcfa0075770 0x7fcfa0077c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.419+0000 7fcf9bfff640 1 -- 172.21.15.133:0/1621766316 <== osd.1 v2:172.21.15.133:6808/198853132 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcfa00782e0 con 0x7fcfa0075770 2024-02-21T23:25:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.423+0000 7f743dffb640 1 -- 172.21.15.133:0/177298284 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7440056e70 con 0x7f744806ede0 2024-02-21T23:25:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.423+0000 7f743dffb640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7f742c074b90 0x7f742c076fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.423+0000 7f743dffb640 1 -- 172.21.15.133:0/177298284 --> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f742c0776a0 con 0x7f742c074b90 2024-02-21T23:25:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.423+0000 7f744cbbb640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7f742c074b90 0x7f742c076fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.424+0000 7f744cbbb640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7f742c074b90 0x7f742c076fd0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.424+0000 7f743dffb640 1 -- 172.21.15.133:0/177298284 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7440055070 con 0x7f744806ede0 2024-02-21T23:25:54.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.425+0000 7f743dffb640 1 -- 172.21.15.133:0/177298284 <== osd.5 v2:172.21.15.142:6818/3321674168 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f742c0776a0 con 0x7f742c074b90 2024-02-21T23:25:54.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.428+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 --> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f746c06a130 con 0x7f7450074ba0 2024-02-21T23:25:54.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f74627fc640 1 -- 172.21.15.133:0/688607958 <== osd.0 v2:172.21.15.133:6800/2680579490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f746c06a130 con 0x7f7450074ba0 2024-02-21T23:25:54.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f7450074ba0 msgr2=0x7f7450076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f7450074ba0 0x7f7450076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f745006b520 msgr2=0x7f745006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f745006b520 0x7f745006d9e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7468036000 tx=0x7f7468030000 comp rx=0 tx=0).stop 2024-02-21T23:25:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 msgr2=0x7f746c0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 --2- 172.21.15.133:0/688607958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746c154600 0x7f746c0cdd20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7464007e40 tx=0x7f746400e4b0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7472014640 1 -- 172.21.15.133:0/688607958 reap_dead start 2024-02-21T23:25:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 shutdown_connections 2024-02-21T23:25:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 >> 172.21.15.133:0/688607958 conn(0x7f746c062000 msgr2=0x7f746c149d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 shutdown_connections 2024-02-21T23:25:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.429+0000 7f7473a77640 1 -- 172.21.15.133:0/688607958 wait complete. 2024-02-21T23:25:54.426 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:54.426 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-02-21T23:25:54.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.444+0000 7fc4f4ff9640 1 -- 172.21.15.133:0/3291839298 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4f0009580 con 0x7fc4f8154750 2024-02-21T23:25:54.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.444+0000 7fc4f4ff9640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc4e406b4d0 0x7fc4e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.444+0000 7fc4f7fff640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc4e406b4d0 0x7fc4e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.445+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 --> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcfb4068a30 con 0x7fcfa0075770 2024-02-21T23:25:54.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.445+0000 7fc4f7fff640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc4e406b4d0 0x7fc4e406d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8037000 tx=0x7fc4e8031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.445+0000 7fcf9bfff640 1 -- 172.21.15.133:0/1621766316 <== osd.1 v2:172.21.15.133:6808/198853132 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcfb4068a30 con 0x7fcfa0075770 2024-02-21T23:25:54.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fcfa0075770 msgr2=0x7fcfa0077c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fcfa0075770 0x7fcfa0077c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcfa006b500 msgr2=0x7fcfa006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcfa006b500 0x7fcfa006d9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40c6660 tx=0x7fcfa4007a40 comp rx=0 tx=0).stop 2024-02-21T23:25:54.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb4070ee0 msgr2=0x7fcfb4158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 --2- 172.21.15.133:0/1621766316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb4070ee0 0x7fcfb4158bd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009e80 tx=0x7fcfac004a40 comp rx=0 tx=0).stop 2024-02-21T23:25:54.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb3fff640 1 -- 172.21.15.133:0/1621766316 reap_dead start 2024-02-21T23:25:54.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 shutdown_connections 2024-02-21T23:25:54.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 >> 172.21.15.133:0/1621766316 conn(0x7fcfb40cb5f0 msgr2=0x7fcfb40c8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 shutdown_connections 2024-02-21T23:25:54.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.446+0000 7fcfb8f8e640 1 -- 172.21.15.133:0/1621766316 wait complete. 2024-02-21T23:25:54.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.449+0000 7fc4f4ff9640 1 -- 172.21.15.133:0/3291839298 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc4f0059050 con 0x7fc4f8154750 2024-02-21T23:25:54.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.450+0000 7fc4f4ff9640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4e4074ac0 0x7fc4e4076f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.450+0000 7fc4f4ff9640 1 -- 172.21.15.133:0/3291839298 --> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4f0014ec0 con 0x7fc4e4074ac0 2024-02-21T23:25:54.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.450+0000 7fc4f77fe640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4e4074ac0 0x7fc4e4076f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.450+0000 7fc4f77fe640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4e4074ac0 0x7fc4e4076f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.451+0000 7fc4f4ff9640 1 -- 172.21.15.133:0/3291839298 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4f0057250 con 0x7fc4f8154750 2024-02-21T23:25:54.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.451+0000 7fc4f4ff9640 1 -- 172.21.15.133:0/3291839298 <== osd.3 v2:172.21.15.133:6824/4089028447 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc4f0014ec0 con 0x7fc4e4074ac0 2024-02-21T23:25:54.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.454+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 --> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7448153600 con 0x7f742c074b90 2024-02-21T23:25:54.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.455+0000 7f743dffb640 1 -- 172.21.15.133:0/177298284 <== osd.5 v2:172.21.15.142:6818/3321674168 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7448153600 con 0x7f742c074b90 2024-02-21T23:25:54.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.456+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7f742c074b90 msgr2=0x7f742c076fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.456+0000 7f744f620640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7f742c074b90 0x7f742c076fd0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.456+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f742c06b520 msgr2=0x7f742c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.456+0000 7f744f620640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f742c06b520 0x7f742c06d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7438036000 tx=0x7f7438030000 comp rx=0 tx=0).stop 2024-02-21T23:25:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.456+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806ede0 msgr2=0x7f7448149380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.456+0000 7f744f620640 1 --2- 172.21.15.133:0/177298284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f744806ede0 0x7f7448149380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f744000b9a0 tx=0x7f744000be60 comp rx=0 tx=0).stop 2024-02-21T23:25:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.456+0000 7f744dbbd640 1 -- 172.21.15.133:0/177298284 reap_dead start 2024-02-21T23:25:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.456+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 shutdown_connections 2024-02-21T23:25:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.456+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 >> 172.21.15.133:0/177298284 conn(0x7f74480cb250 msgr2=0x7f7448152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.457+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 shutdown_connections 2024-02-21T23:25:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.457+0000 7f744f620640 1 -- 172.21.15.133:0/177298284 wait complete. 2024-02-21T23:25:54.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.460+0000 7fcd20ff9640 1 -- 172.21.15.133:0/3861157775 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd24094480 con 0x7fcd1c0e6910 2024-02-21T23:25:54.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.461+0000 7fcd20ff9640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcd0806b4d0 0x7fcd0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.447 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:54.448 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-02-21T23:25:54.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.461+0000 7fcd23fff640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcd0806b4d0 0x7fcd0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.462+0000 7fcd23fff640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcd0806b4d0 0x7fcd0806d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcd18005880 tx=0x7fcd18009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.465+0000 7fcd20ff9640 1 -- 172.21.15.133:0/3861157775 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcd240d25d0 con 0x7fcd1c0e6910 2024-02-21T23:25:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.465+0000 7fcd20ff9640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fcd08074ad0 0x7fcd08076f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.465+0000 7fcd20ff9640 1 -- 172.21.15.133:0/3861157775 --> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd240a2e50 con 0x7fcd08074ad0 2024-02-21T23:25:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.465+0000 7fcd237fe640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fcd08074ad0 0x7fcd08076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.466+0000 7fcd237fe640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fcd08074ad0 0x7fcd08076f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.466+0000 7fcd20ff9640 1 -- 172.21.15.133:0/3861157775 <== osd.6 v2:172.21.15.142:6802/4241695382 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcd240a2e50 con 0x7fcd08074ad0 2024-02-21T23:25:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.468+0000 7fcd20ff9640 1 -- 172.21.15.133:0/3861157775 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd240d07d0 con 0x7fcd1c0e6910 2024-02-21T23:25:54.456 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:54.456 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-02-21T23:25:54.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.481+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 --> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc4f806a130 con 0x7fc4e4074ac0 2024-02-21T23:25:54.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.482+0000 7fc4f4ff9640 1 -- 172.21.15.133:0/3291839298 <== osd.3 v2:172.21.15.133:6824/4089028447 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc4f806a130 con 0x7fc4e4074ac0 2024-02-21T23:25:54.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.482+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4e4074ac0 msgr2=0x7fc4e4076f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.482+0000 7fc4fe3bb640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4e4074ac0 0x7fc4e4076f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc4e406b4d0 msgr2=0x7fc4e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7fc4fe3bb640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc4e406b4d0 0x7fc4e406d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8037000 tx=0x7fc4e8031000 comp rx=0 tx=0).stop 2024-02-21T23:25:54.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8154750 msgr2=0x7fc4f8145ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7fc4fe3bb640 1 --2- 172.21.15.133:0/3291839298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8154750 0x7fc4f8145ce0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000df40 tx=0x7fc4f000df70 comp rx=0 tx=0).stop 2024-02-21T23:25:54.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7fc4fc958640 1 -- 172.21.15.133:0/3291839298 reap_dead start 2024-02-21T23:25:54.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 shutdown_connections 2024-02-21T23:25:54.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 >> 172.21.15.133:0/3291839298 conn(0x7fc4f806f6e0 msgr2=0x7fc4f8155510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.482+0000 7f98d17fa640 1 -- 172.21.15.133:0/2530679820 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98c801a070 con 0x7f98cc04ba50 2024-02-21T23:25:54.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7f98d17fa640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98b806b550 0x7f98b806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 shutdown_connections 2024-02-21T23:25:54.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7fc4fe3bb640 1 -- 172.21.15.133:0/3291839298 wait complete. 2024-02-21T23:25:54.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.483+0000 7f98d3fff640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98b806b550 0x7f98b806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.484+0000 7f98d3fff640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98b806b550 0x7f98b806da10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98cc04c9f0 tx=0x7f98c00090e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.486+0000 7f98d17fa640 1 -- 172.21.15.133:0/2530679820 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f98c8013e10 con 0x7f98cc04ba50 2024-02-21T23:25:54.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.487+0000 7f98d17fa640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7f98b8074bd0 0x7f98b8077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.487+0000 7f98d17fa640 1 -- 172.21.15.133:0/2530679820 --> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98c8020ec0 con 0x7f98b8074bd0 2024-02-21T23:25:54.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.487+0000 7f98d9121640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7f98b8074bd0 0x7f98b8077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.488+0000 7f98d9121640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7f98b8074bd0 0x7f98b8077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.489+0000 7f98d17fa640 1 -- 172.21.15.133:0/2530679820 <== osd.4 v2:172.21.15.142:6826/2259485122 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f98c8020ec0 con 0x7f98b8074bd0 2024-02-21T23:25:54.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.489+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 --> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcd04002d10 con 0x7fcd08074ad0 2024-02-21T23:25:54.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.490+0000 7fcd20ff9640 1 -- 172.21.15.133:0/3861157775 <== osd.6 v2:172.21.15.142:6802/4241695382 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcd04002d10 con 0x7fcd08074ad0 2024-02-21T23:25:54.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.490+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fcd08074ad0 msgr2=0x7fcd08076f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.490+0000 7fcd293d4640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fcd08074ad0 0x7fcd08076f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.492+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcd0806b4d0 msgr2=0x7fcd0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.492+0000 7fcd293d4640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcd0806b4d0 0x7fcd0806d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcd18005880 tx=0x7fcd18009040 comp rx=0 tx=0).stop 2024-02-21T23:25:54.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.492+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 msgr2=0x7fcd1c16d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.492+0000 7fcd293d4640 1 --2- 172.21.15.133:0/3861157775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd1c0e6910 0x7fcd1c16d120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd24064a20 tx=0x7fcd24068aa0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.493+0000 7fcd28bd3640 1 -- 172.21.15.133:0/3861157775 reap_dead start 2024-02-21T23:25:54.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.493+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 shutdown_connections 2024-02-21T23:25:54.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.493+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 >> 172.21.15.133:0/3861157775 conn(0x7fcd1c006190 msgr2=0x7fcd1c0e18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.493+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 shutdown_connections 2024-02-21T23:25:54.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.493+0000 7fcd293d4640 1 -- 172.21.15.133:0/3861157775 wait complete. 2024-02-21T23:25:54.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.495+0000 7f98d17fa640 1 -- 172.21.15.133:0/2530679820 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98c8054640 con 0x7f98cc04ba50 2024-02-21T23:25:54.489 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:54.489 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-02-21T23:25:54.494 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:54.494 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-02-21T23:25:54.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.511+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 --> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f98cc04b830 con 0x7f98b8074bd0 2024-02-21T23:25:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98d17fa640 1 -- 172.21.15.133:0/2530679820 <== osd.4 v2:172.21.15.142:6826/2259485122 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f98cc04b830 con 0x7f98b8074bd0 2024-02-21T23:25:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7f98b8074bd0 msgr2=0x7f98b8077010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98dab84640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7f98b8074bd0 0x7f98b8077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98b806b550 msgr2=0x7f98b806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98dab84640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98b806b550 0x7f98b806da10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98cc04c9f0 tx=0x7f98c00090e0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98cc04ba50 msgr2=0x7f98cc0d4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98dab84640 1 --2- 172.21.15.133:0/2530679820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98cc04ba50 0x7f98cc0d4890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c390 tx=0x7f98c800c850 comp rx=0 tx=0).stop 2024-02-21T23:25:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98d9121640 1 -- 172.21.15.133:0/2530679820 reap_dead start 2024-02-21T23:25:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 shutdown_connections 2024-02-21T23:25:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.512+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 >> 172.21.15.133:0/2530679820 conn(0x7f98cc050170 msgr2=0x7f98cc0e2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.513+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 shutdown_connections 2024-02-21T23:25:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.513+0000 7f98dab84640 1 -- 172.21.15.133:0/2530679820 wait complete. 2024-02-21T23:25:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.519+0000 7f16127fc640 1 -- 172.21.15.133:0/202493326 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1618018020 con 0x7f161c0c7c10 2024-02-21T23:25:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.519+0000 7f16127fc640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f160006b500 0x7f160006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.520+0000 7f1621db8640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f160006b500 0x7f160006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.520+0000 7f1621db8640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f160006b500 0x7f160006d9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1614001ef0 tx=0x7f1614009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.524+0000 7f16127fc640 1 -- 172.21.15.133:0/202493326 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1618057100 con 0x7f161c0c7c10 2024-02-21T23:25:54.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.524+0000 7f16127fc640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f1600074b00 0x7f1600076f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.524+0000 7f16127fc640 1 -- 172.21.15.133:0/202493326 --> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f161800de50 con 0x7f1600074b00 2024-02-21T23:25:54.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.525+0000 7f16127fc640 1 -- 172.21.15.133:0/202493326 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1618055300 con 0x7f161c0c7c10 2024-02-21T23:25:54.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.525+0000 7f16225b9640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f1600074b00 0x7f1600076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.526+0000 7f16225b9640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f1600074b00 0x7f1600076f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.527+0000 7f16127fc640 1 -- 172.21.15.133:0/202493326 <== osd.2 v2:172.21.15.133:6816/3824311867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f161800de50 con 0x7f1600074b00 2024-02-21T23:25:54.516 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:54.517 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-02-21T23:25:54.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.557+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 --> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f161c06a130 con 0x7f1600074b00 2024-02-21T23:25:54.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.557+0000 7f16127fc640 1 -- 172.21.15.133:0/202493326 <== osd.2 v2:172.21.15.133:6816/3824311867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f161c06a130 con 0x7f1600074b00 2024-02-21T23:25:54.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.558+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f1600074b00 msgr2=0x7f1600076f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.558+0000 7f162401c640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f1600074b00 0x7f1600076f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.560+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f160006b500 msgr2=0x7f160006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.560+0000 7f162401c640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f160006b500 0x7f160006d9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1614001ef0 tx=0x7f1614009040 comp rx=0 tx=0).stop 2024-02-21T23:25:54.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.560+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c7c10 msgr2=0x7f161c159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.560+0000 7f162401c640 1 --2- 172.21.15.133:0/202493326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f161c0c7c10 0x7f161c159320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f161800b440 tx=0x7f161800b900 comp rx=0 tx=0).stop 2024-02-21T23:25:54.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.560+0000 7f16225b9640 1 -- 172.21.15.133:0/202493326 reap_dead start 2024-02-21T23:25:54.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.560+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 shutdown_connections 2024-02-21T23:25:54.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.560+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 >> 172.21.15.133:0/202493326 conn(0x7f161c062000 msgr2=0x7f161c0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.560+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 shutdown_connections 2024-02-21T23:25:54.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.561+0000 7f162401c640 1 -- 172.21.15.133:0/202493326 wait complete. 2024-02-21T23:25:54.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.565+0000 7f3fe7ed5640 1 Processor -- start 2024-02-21T23:25:54.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.566+0000 7f3fe7ed5640 1 -- start start 2024-02-21T23:25:54.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.567+0000 7f3fe7ed5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe014ec20 0x7f3fe00c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.567+0000 7f3fe7ed5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00717e0 con 0x7f3fe00ce3c0 2024-02-21T23:25:54.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.567+0000 7f3fe7ed5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe0158090 con 0x7f3fe00cdfe0 2024-02-21T23:25:54.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.567+0000 7f3fe7ed5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00c84b0 con 0x7f3fe014ec20 2024-02-21T23:25:54.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.568+0000 7f3fe6472640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe014ec20 0x7f3fe00c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.569+0000 7f3fe6472640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe014ec20 0x7f3fe00c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45840/0 (socket says 172.21.15.133:45840) 2024-02-21T23:25:54.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.569+0000 7f3fe6472640 1 -- 172.21.15.133:0/2860245297 learned_addr learned my addr 172.21.15.133:0/2860245297 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.574+0000 7f3fe6472640 1 -- 172.21.15.133:0/2860245297 >> v1:172.21.15.142:6789/0 conn(0x7f3fe00cdfe0 legacy=0x7f3fe0070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.574+0000 7f3fe6472640 1 -- 172.21.15.133:0/2860245297 >> v1:172.21.15.133:6789/0 conn(0x7f3fe00ce3c0 legacy=0x7f3fe014e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.574+0000 7f3fe6472640 1 -- 172.21.15.133:0/2860245297 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe0158370 con 0x7f3fe014ec20 2024-02-21T23:25:54.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.579+0000 7f3fe6472640 1 --2- 172.21.15.133:0/2860245297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe014ec20 0x7f3fe00c3250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009e80 tx=0x7f3fd802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4423bcbf7bebda59 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.567 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:54.567 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-02-21T23:25:54.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.583+0000 7f3fe4c6f640 1 -- 172.21.15.133:0/2860245297 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd802cb90 con 0x7f3fe014ec20 2024-02-21T23:25:54.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.583+0000 7f3fe4c6f640 1 -- 172.21.15.133:0/2860245297 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd802cd30 con 0x7f3fe014ec20 2024-02-21T23:25:54.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.583+0000 7f3fe4c6f640 1 -- 172.21.15.133:0/2860245297 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8034610 con 0x7f3fe014ec20 2024-02-21T23:25:54.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.584+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/2860245297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe014ec20 msgr2=0x7f3fe00c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.584+0000 7f3fe7ed5640 1 --2- 172.21.15.133:0/2860245297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe014ec20 0x7f3fe00c3250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009e80 tx=0x7f3fd802bff0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.584+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/2860245297 shutdown_connections 2024-02-21T23:25:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.584+0000 7f3fe7ed5640 1 --2- 172.21.15.133:0/2860245297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe014ec20 0x7f3fe00c3250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.584+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/2860245297 >> 172.21.15.133:0/2860245297 conn(0x7f3fe006aec0 msgr2=0x7f3fe006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.584+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/2860245297 shutdown_connections 2024-02-21T23:25:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.584+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/2860245297 wait complete. 2024-02-21T23:25:54.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.586+0000 7f3fe7ed5640 1 Processor -- start 2024-02-21T23:25:54.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.586+0000 7f3fe7ed5640 1 -- start start 2024-02-21T23:25:54.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.587+0000 7f3fe7ed5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe00cdfe0 0x7f3fe014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.587+0000 7f3fe7ed5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe00ce3c0 0x7f3fe01ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.587+0000 7f3fe7ed5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe014ec20 0x7f3fe01efa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.587+0000 7f3fe7ed5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0158930 con 0x7f3fe00cdfe0 2024-02-21T23:25:54.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.587+0000 7f3fe7ed5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0158ca0 con 0x7f3fe014ec20 2024-02-21T23:25:54.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.587+0000 7f3fe7ed5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0071480 con 0x7f3fe00ce3c0 2024-02-21T23:25:54.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.588+0000 7f3fe6472640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe014ec20 0x7f3fe01efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.590+0000 7f3fe6472640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe014ec20 0x7f3fe01efa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42636/0 (socket says 172.21.15.133:42636) 2024-02-21T23:25:54.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.590+0000 7f3fe6472640 1 -- 172.21.15.133:0/1018913996 learned_addr learned my addr 172.21.15.133:0/1018913996 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.590+0000 7f3fe5470640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe00ce3c0 0x7f3fe01ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.593+0000 7f3fe6472640 1 -- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe00ce3c0 msgr2=0x7f3fe01ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.593+0000 7f3fe6472640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe00ce3c0 0x7f3fe01ef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.593+0000 7f3fe6472640 1 -- 172.21.15.133:0/1018913996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe00cdfe0 msgr2=0x7f3fe014b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.593+0000 7f3fe6472640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe00cdfe0 0x7f3fe014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.593+0000 7f3fe6472640 1 -- 172.21.15.133:0/1018913996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe0154ad0 con 0x7f3fe014ec20 2024-02-21T23:25:54.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.598+0000 7f3fe6472640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe014ec20 0x7f3fe01efa30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009fa0 tx=0x7f3fd8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.600+0000 7f1e456d5640 1 Processor -- start 2024-02-21T23:25:54.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.601+0000 7f1e456d5640 1 -- start start 2024-02-21T23:25:54.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.602+0000 7f3fd67fc640 1 -- 172.21.15.133:0/1018913996 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd803e650 con 0x7f3fe014ec20 2024-02-21T23:25:54.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.602+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe0154db0 con 0x7f3fe014ec20 2024-02-21T23:25:54.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.602+0000 7f1e456d5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e401588f0 0x7f1e40161c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.602+0000 7f1e456d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c64b0 0x7f1e400c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.602+0000 7f1e456d5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e40162b70 con 0x7f1e400c7820 2024-02-21T23:25:54.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.602+0000 7f1e456d5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e401622d0 con 0x7f1e401588f0 2024-02-21T23:25:54.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.602+0000 7f1e456d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401625b0 con 0x7f1e400c64b0 2024-02-21T23:25:54.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.602+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe0155370 con 0x7f3fe014ec20 2024-02-21T23:25:54.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.602+0000 7f3fd67fc640 1 -- 172.21.15.133:0/1018913996 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd803e810 con 0x7f3fe014ec20 2024-02-21T23:25:54.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.603+0000 7f1e3f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c64b0 0x7f1e400c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.603+0000 7f3fd67fc640 1 -- 172.21.15.133:0/1018913996 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd803ec00 con 0x7f3fe014ec20 2024-02-21T23:25:54.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.604+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc8005290 con 0x7f3fe014ec20 2024-02-21T23:25:54.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.603+0000 7f1e3effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e401588f0 0x7f1e40161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.603+0000 7f1e3effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e401588f0 0x7f1e40161c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40072/0 (socket says 172.21.15.133:40072) 2024-02-21T23:25:54.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.604+0000 7f1e3effd640 1 -- 172.21.15.133:0/4122059644 learned_addr learned my addr 172.21.15.133:0/4122059644 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.606+0000 7f1e3effd640 1 -- 172.21.15.133:0/4122059644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c64b0 msgr2=0x7f1e400c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.606+0000 7f1e3effd640 1 --2- 172.21.15.133:0/4122059644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c64b0 0x7f1e400c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.606+0000 7f1e3effd640 1 -- 172.21.15.133:0/4122059644 >> v1:172.21.15.142:6789/0 conn(0x7f1e400c7820 legacy=0x7f1e4015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.606+0000 7f1e3effd640 1 -- 172.21.15.133:0/4122059644 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40162890 con 0x7f1e401588f0 2024-02-21T23:25:54.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.610+0000 7f1e3effd640 1 --2- 172.21.15.133:0/4122059644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e401588f0 0x7f1e40161c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009870 tx=0x7f1e3002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e91c11b6bf44c0d server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.616+0000 7f1e3e7fc640 1 -- 172.21.15.133:0/4122059644 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30034070 con 0x7f1e401588f0 2024-02-21T23:25:54.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.616+0000 7f1e3e7fc640 1 -- 172.21.15.133:0/4122059644 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30030840 con 0x7f1e401588f0 2024-02-21T23:25:54.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.617+0000 7f1e3e7fc640 1 -- 172.21.15.133:0/4122059644 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30030c10 con 0x7f1e401588f0 2024-02-21T23:25:54.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.617+0000 7f1e456d5640 1 -- 172.21.15.133:0/4122059644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e401588f0 msgr2=0x7f1e40161c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.617+0000 7f1e456d5640 1 --2- 172.21.15.133:0/4122059644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e401588f0 0x7f1e40161c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009870 tx=0x7f1e3002b980 comp rx=0 tx=0).stop 2024-02-21T23:25:54.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.617+0000 7f1e456d5640 1 -- 172.21.15.133:0/4122059644 shutdown_connections 2024-02-21T23:25:54.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.617+0000 7f1e456d5640 1 --2- 172.21.15.133:0/4122059644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c64b0 0x7f1e400c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.617+0000 7f1e456d5640 1 --2- 172.21.15.133:0/4122059644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e401588f0 0x7f1e40161c20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.617+0000 7f1e456d5640 1 -- 172.21.15.133:0/4122059644 >> 172.21.15.133:0/4122059644 conn(0x7f1e4006aec0 msgr2=0x7f1e4006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.618+0000 7f1e456d5640 1 -- 172.21.15.133:0/4122059644 shutdown_connections 2024-02-21T23:25:54.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.618+0000 7f1e456d5640 1 -- 172.21.15.133:0/4122059644 wait complete. 2024-02-21T23:25:54.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.620+0000 7f1e456d5640 1 Processor -- start 2024-02-21T23:25:54.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.620+0000 7f1e456d5640 1 -- start start 2024-02-21T23:25:54.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.621+0000 7f1e456d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e400c64b0 0x7f1e400c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.621+0000 7f1e456d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c7820 0x7f1e400ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.621+0000 7f1e456d5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e401588f0 0x7f1e40161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.621+0000 7f1e456d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40158cd0 con 0x7f1e400c64b0 2024-02-21T23:25:54.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.621+0000 7f1e456d5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4015eab0 con 0x7f1e401588f0 2024-02-21T23:25:54.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.621+0000 7f1e456d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e400715a0 con 0x7f1e400c7820 2024-02-21T23:25:54.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.621+0000 7fd195399640 1 Processor -- start 2024-02-21T23:25:54.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.622+0000 7fd195399640 1 -- start start 2024-02-21T23:25:54.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.622+0000 7f1e37fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c7820 0x7f1e400ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.622+0000 7f1e3effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e400c64b0 0x7f1e400c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.622+0000 7f1e37fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c7820 0x7f1e400ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45864/0 (socket says 172.21.15.133:45864) 2024-02-21T23:25:54.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.622+0000 7f1e37fff640 1 -- 172.21.15.133:0/3513044872 learned_addr learned my addr 172.21.15.133:0/3513044872 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.623+0000 7fd195399640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1900c7f30 0x7fd19015e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.623+0000 7fd195399640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd190158ab0 con 0x7fd1901586d0 2024-02-21T23:25:54.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.623+0000 7fd195399640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19015f330 con 0x7fd1900c65c0 2024-02-21T23:25:54.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.623+0000 7fd195399640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd190162980 con 0x7fd1900c7f30 2024-02-21T23:25:54.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.624+0000 7f3fd67fc640 1 -- 172.21.15.133:0/1018913996 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fd802cba0 con 0x7f3fe014ec20 2024-02-21T23:25:54.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.624+0000 7f3fd67fc640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3fc406b520 0x7f3fc406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.624+0000 7f3fe5c71640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3fc406b520 0x7f3fc406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.625+0000 7f3fe5c71640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3fc406b520 0x7f3fc406d9e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0000f60 tx=0x7f3fd00079b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.625+0000 7fd18f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1900c7f30 0x7fd19015e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.626+0000 7f3fd67fc640 1 -- 172.21.15.133:0/1018913996 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3fd804ae70 con 0x7f3fe014ec20 2024-02-21T23:25:54.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.630+0000 7f1e3effd640 1 -- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c7820 msgr2=0x7f1e400ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.630+0000 7f1e3effd640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c7820 0x7f1e400ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.630+0000 7f1e3effd640 1 -- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e401588f0 msgr2=0x7f1e40161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.630+0000 7f1e3effd640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e401588f0 0x7f1e40161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.630+0000 7f1e3effd640 1 -- 172.21.15.133:0/3513044872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e400cc060 con 0x7f1e400c64b0 2024-02-21T23:25:54.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.631+0000 7fd18effd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd1901586d0 0x7fd190162270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51398/0 (socket says 172.21.15.133:51398) 2024-02-21T23:25:54.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.631+0000 7fd18effd640 1 -- 172.21.15.133:0/2113543636 learned_addr learned my addr 172.21.15.133:0/2113543636 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:54.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.638+0000 7fd18dffb640 1 -- 172.21.15.133:0/2113543636 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366006788 0 0) 0x7fd190158ab0 con 0x7fd1901586d0 2024-02-21T23:25:54.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.638+0000 7fd18dffb640 1 -- 172.21.15.133:0/2113543636 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd178003520 con 0x7fd1901586d0 2024-02-21T23:25:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.640+0000 7fd18f7fe640 1 -- 172.21.15.133:0/2113543636 >> v1:172.21.15.142:6789/0 conn(0x7fd1900c65c0 legacy=0x7fd1900cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.640+0000 7fd18f7fe640 1 -- 172.21.15.133:0/2113543636 >> v1:172.21.15.133:6789/0 conn(0x7fd1901586d0 legacy=0x7fd190162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.640+0000 7fd18f7fe640 1 -- 172.21.15.133:0/2113543636 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19015ea90 con 0x7fd1900c7f30 2024-02-21T23:25:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.647+0000 7fd18f7fe640 1 --2- 172.21.15.133:0/2113543636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900c7f30 0x7fd19015e3e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd1840097a0 tx=0x7fd18402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc70024c49ac502 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.648+0000 7f1e3effd640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e400c64b0 0x7f1e400c3eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e30033000 tx=0x7f1e300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.655+0000 7f46318d1640 1 Processor -- start 2024-02-21T23:25:54.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.655+0000 7f46318d1640 1 -- start start 2024-02-21T23:25:54.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.656+0000 7f46318d1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f462c0cd010 0x7f462c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.656+0000 7f46318d1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462c156e90 con 0x7f462c0c41a0 2024-02-21T23:25:54.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.656+0000 7f46318d1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462c157150 con 0x7f462c0c3dc0 2024-02-21T23:25:54.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.656+0000 7f46318d1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462c0cd4c0 con 0x7f462c0cd010 2024-02-21T23:25:54.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.658+0000 7f1e377fe640 1 -- 172.21.15.133:0/3513044872 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30034040 con 0x7f1e400c64b0 2024-02-21T23:25:54.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.659+0000 7f1e377fe640 1 -- 172.21.15.133:0/3513044872 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30039d90 con 0x7f1e400c64b0 2024-02-21T23:25:54.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.659+0000 7f1e377fe640 1 -- 172.21.15.133:0/3513044872 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30043650 con 0x7f1e400c64b0 2024-02-21T23:25:54.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.659+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e400cc340 con 0x7f1e400c64b0 2024-02-21T23:25:54.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.659+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e400cb620 con 0x7f1e400c64b0 2024-02-21T23:25:54.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.660+0000 7fd18dffb640 1 -- 172.21.15.133:0/2113543636 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184034070 con 0x7fd1900c7f30 2024-02-21T23:25:54.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.660+0000 7fd18dffb640 1 -- 172.21.15.133:0/2113543636 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184030610 con 0x7fd1900c7f30 2024-02-21T23:25:54.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.660+0000 7fd18dffb640 1 -- 172.21.15.133:0/2113543636 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1840309e0 con 0x7fd1900c7f30 2024-02-21T23:25:54.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.660+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4006a130 con 0x7f1e400c64b0 2024-02-21T23:25:54.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.660+0000 7f462b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f462c0cd010 0x7f462c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.661+0000 7fd195399640 1 -- 172.21.15.133:0/2113543636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900c7f30 msgr2=0x7fd19015e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.661+0000 7fd195399640 1 --2- 172.21.15.133:0/2113543636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900c7f30 0x7fd19015e3e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd1840097a0 tx=0x7fd18402b960 comp rx=0 tx=0).stop 2024-02-21T23:25:54.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.661+0000 7fd195399640 1 -- 172.21.15.133:0/2113543636 shutdown_connections 2024-02-21T23:25:54.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.661+0000 7fd195399640 1 --2- 172.21.15.133:0/2113543636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900c7f30 0x7fd19015e3e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.661+0000 7fd195399640 1 -- 172.21.15.133:0/2113543636 >> 172.21.15.133:0/2113543636 conn(0x7fd19006bac0 msgr2=0x7fd19006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.661+0000 7fd195399640 1 -- 172.21.15.133:0/2113543636 shutdown_connections 2024-02-21T23:25:54.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.661+0000 7fd195399640 1 -- 172.21.15.133:0/2113543636 wait complete. 2024-02-21T23:25:54.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.661+0000 7f462b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f462c0cd010 0x7f462c15a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40098/0 (socket says 172.21.15.133:40098) 2024-02-21T23:25:54.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.661+0000 7f462b7fe640 1 -- 172.21.15.133:0/1710479941 learned_addr learned my addr 172.21.15.133:0/1710479941 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.663+0000 7fd195399640 1 Processor -- start 2024-02-21T23:25:54.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.663+0000 7fd195399640 1 -- start start 2024-02-21T23:25:54.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.664+0000 7fd195399640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1900c65c0 0x7fd190161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.664+0000 7fd195399640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900c7f30 0x7fd19015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.664+0000 7fd195399640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1901586d0 0x7fd19015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.664+0000 7fd195399640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd190161a70 con 0x7fd1900c7f30 2024-02-21T23:25:54.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.664+0000 7fd195399640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19015f050 con 0x7fd1900c65c0 2024-02-21T23:25:54.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.664+0000 7fd195399640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd190161d50 con 0x7fd1901586d0 2024-02-21T23:25:54.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.665+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1901586d0 0x7fd19015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.665+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1901586d0 0x7fd19015cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45878/0 (socket says 172.21.15.133:45878) 2024-02-21T23:25:54.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.665+0000 7fd18f7fe640 1 -- 172.21.15.133:0/3739592566 learned_addr learned my addr 172.21.15.133:0/3739592566 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.665+0000 7f462b7fe640 1 -- 172.21.15.133:0/1710479941 >> v1:172.21.15.142:6789/0 conn(0x7f462c0c3dc0 legacy=0x7f462c070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.665+0000 7f462b7fe640 1 -- 172.21.15.133:0/1710479941 >> v1:172.21.15.133:6789/0 conn(0x7f462c0c41a0 legacy=0x7f462c150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.665+0000 7f462b7fe640 1 -- 172.21.15.133:0/1710479941 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c157430 con 0x7f462c0cd010 2024-02-21T23:25:54.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.668+0000 7f2bedf1e640 1 Processor -- start 2024-02-21T23:25:54.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.668+0000 7f2bedf1e640 1 -- start start 2024-02-21T23:25:54.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.668+0000 7f0918808640 1 Processor -- start 2024-02-21T23:25:54.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.668+0000 7f0918808640 1 -- start start 2024-02-21T23:25:54.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.669+0000 7f2bedf1e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2be80c41a0 0x7f2be80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.669+0000 7f0918808640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0914072150 0x7f09140725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.669+0000 7f0918808640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09140c2f60 con 0x7f0914156cd0 2024-02-21T23:25:54.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.669+0000 7f0918808640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0914072b00 con 0x7f0914151130 2024-02-21T23:25:54.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.669+0000 7f0918808640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0914157280 con 0x7f0914072150 2024-02-21T23:25:54.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.670+0000 7f2bedf1e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be8156e90 con 0x7f2be8150d30 2024-02-21T23:25:54.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.670+0000 7f2bedf1e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be8157150 con 0x7f2be80c3dc0 2024-02-21T23:25:54.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.670+0000 7f2bedf1e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be8157430 con 0x7f2be80c41a0 2024-02-21T23:25:54.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.670+0000 7fd18f7fe640 1 -- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1900c65c0 msgr2=0x7fd190161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.670+0000 7fd18f7fe640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1900c65c0 0x7fd190161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.670+0000 7fd18f7fe640 1 -- 172.21.15.133:0/3739592566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900c7f30 msgr2=0x7fd19015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.670+0000 7fd18f7fe640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900c7f30 0x7fd19015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.670+0000 7fd18f7fe640 1 -- 172.21.15.133:0/3739592566 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19015d3d0 con 0x7fd1901586d0 2024-02-21T23:25:54.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.673+0000 7fd18f7fe640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1901586d0 0x7fd19015cbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd1840098c0 tx=0x7fd1840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.678+0000 7fd1777fe640 1 -- 172.21.15.133:0/3739592566 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184034040 con 0x7fd1901586d0 2024-02-21T23:25:54.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.678+0000 7fd1777fe640 1 -- 172.21.15.133:0/3739592566 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184039910 con 0x7fd1901586d0 2024-02-21T23:25:54.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.678+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19015d6b0 con 0x7fd1901586d0 2024-02-21T23:25:54.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.678+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1901566b0 con 0x7fd1901586d0 2024-02-21T23:25:54.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.678+0000 7fd1777fe640 1 -- 172.21.15.133:0/3739592566 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184039d00 con 0x7fd1901586d0 2024-02-21T23:25:54.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.679+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19006a130 con 0x7fd1901586d0 2024-02-21T23:25:54.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.700+0000 7fd1777fe640 1 -- 172.21.15.133:0/3739592566 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd184037070 con 0x7fd1901586d0 2024-02-21T23:25:54.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.700+0000 7fd1777fe640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd17c06b4d0 0x7fd17c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.701+0000 7fd18effd640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd17c06b4d0 0x7fd17c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.701+0000 7fd18effd640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd17c06b4d0 0x7fd17c06d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd188001570 tx=0x7fd188007930 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.701+0000 7f2e4e51e640 1 Processor -- start 2024-02-21T23:25:54.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.701+0000 7f2e4e51e640 1 -- start start 2024-02-21T23:25:54.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.702+0000 7f2e4e51e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2e48154240 0x7f2e480c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.702+0000 7f2e4e51e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4815d630 0x7f2e4815da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.702+0000 7f2e4e51e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4815dfc0 con 0x7f2e480c8420 2024-02-21T23:25:54.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.702+0000 7f2e4e51e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e480c3ff0 con 0x7f2e48154240 2024-02-21T23:25:54.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.702+0000 7f2e4e51e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480c8990 con 0x7f2e4815d630 2024-02-21T23:25:54.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.702+0000 7fd1777fe640 1 -- 172.21.15.133:0/3739592566 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd1840793f0 con 0x7fd1901586d0 2024-02-21T23:25:54.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.712+0000 7f59c8636640 1 Processor -- start 2024-02-21T23:25:54.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.712+0000 7f59c8636640 1 -- start start 2024-02-21T23:25:54.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.713+0000 7f59c8636640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f59c00ce3c0 0x7f59c00c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.713+0000 7f59c8636640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f59c014e510 0x7f59c014e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.714+0000 7f59c8636640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c014eea0 con 0x7f59c00cdfe0 2024-02-21T23:25:54.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.714+0000 7f59c8636640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c00c84b0 con 0x7f59c014e510 2024-02-21T23:25:54.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.714+0000 7f59c8636640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c0158ca0 con 0x7f59c00ce3c0 2024-02-21T23:25:54.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.867+0000 7f2be6ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2be80c41a0 0x7f2be80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.867+0000 7f2be6ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2be80c41a0 0x7f2be80cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42676/0 (socket says 172.21.15.133:42676) 2024-02-21T23:25:54.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.867+0000 7f2be6ffd640 1 -- 172.21.15.133:0/2553005288 learned_addr learned my addr 172.21.15.133:0/2553005288 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.872+0000 7f2be6ffd640 1 -- 172.21.15.133:0/2553005288 >> v1:172.21.15.142:6789/0 conn(0x7f2be80c3dc0 legacy=0x7f2be8070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.872+0000 7f2be6ffd640 1 -- 172.21.15.133:0/2553005288 >> v1:172.21.15.133:6789/0 conn(0x7f2be8150d30 legacy=0x7f2be815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.872+0000 7f2be6ffd640 1 -- 172.21.15.133:0/2553005288 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be8157710 con 0x7f2be80c41a0 2024-02-21T23:25:54.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.872+0000 7f2e47fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2e48154240 0x7f2e480c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.874+0000 7f2e47fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2e48154240 0x7f2e480c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42678/0 (socket says 172.21.15.133:42678) 2024-02-21T23:25:54.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.874+0000 7f2e47fff640 1 -- 172.21.15.133:0/320670964 learned_addr learned my addr 172.21.15.133:0/320670964 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.892+0000 7f3fd67fc640 1 -- 172.21.15.133:0/1018913996 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f3fd807fa90 con 0x7f3fe014ec20 2024-02-21T23:25:54.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.892+0000 7f2be6ffd640 1 --2- 172.21.15.133:0/2553005288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 0x7f2be80cd010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecc1fa44b3272a4a server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.894+0000 7f2e47fff640 1 -- 172.21.15.133:0/320670964 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4815d630 msgr2=0x7f2e4815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.894+0000 7f2e47fff640 1 --2- 172.21.15.133:0/320670964 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4815d630 0x7f2e4815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.894+0000 7f2e47fff640 1 -- 172.21.15.133:0/320670964 >> v1:172.21.15.133:6789/0 conn(0x7f2e480c8420 legacy=0x7f2e480cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.894+0000 7f2e47fff640 1 -- 172.21.15.133:0/320670964 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e480c8c50 con 0x7f2e48154240 2024-02-21T23:25:54.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.901+0000 7f59c5bd1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f59c00ce3c0 0x7f59c00c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.903+0000 7f2e47fff640 1 --2- 172.21.15.133:0/320670964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 0x7f2e480c9bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009870 tx=0x7f2e3802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70efe0656562e77 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.903+0000 7f59c5bd1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f59c00ce3c0 0x7f59c00c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42684/0 (socket says 172.21.15.133:42684) 2024-02-21T23:25:54.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.903+0000 7f59c5bd1640 1 -- 172.21.15.133:0/794784857 learned_addr learned my addr 172.21.15.133:0/794784857 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.908+0000 7f462b7fe640 1 --2- 172.21.15.133:0/1710479941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 0x7f462c15a600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4624009e50 tx=0x7f462402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb3a8a4b13f0304 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.908+0000 7f2be67fc640 1 -- 172.21.15.133:0/2553005288 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc035020 con 0x7f2be80c41a0 2024-02-21T23:25:54.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.908+0000 7f2be67fc640 1 -- 172.21.15.133:0/2553005288 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc031870 con 0x7f2be80c41a0 2024-02-21T23:25:54.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.908+0000 7f2be67fc640 1 -- 172.21.15.133:0/2553005288 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc031c40 con 0x7f2be80c41a0 2024-02-21T23:25:54.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.908+0000 7f2bedf1e640 1 -- 172.21.15.133:0/2553005288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 msgr2=0x7f2be80cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.908+0000 7f2bedf1e640 1 --2- 172.21.15.133:0/2553005288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 0x7f2be80cd010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc02b910 comp rx=0 tx=0).stop 2024-02-21T23:25:54.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.909+0000 7f2bedf1e640 1 -- 172.21.15.133:0/2553005288 shutdown_connections 2024-02-21T23:25:54.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.909+0000 7f2bedf1e640 1 --2- 172.21.15.133:0/2553005288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 0x7f2be80cd010 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.909+0000 7f2bedf1e640 1 -- 172.21.15.133:0/2553005288 >> 172.21.15.133:0/2553005288 conn(0x7f2be8061b50 msgr2=0x7f2be8072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.909+0000 7f2bedf1e640 1 -- 172.21.15.133:0/2553005288 shutdown_connections 2024-02-21T23:25:54.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.909+0000 7f2bedf1e640 1 -- 172.21.15.133:0/2553005288 wait complete. 2024-02-21T23:25:54.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.910+0000 7f2bedf1e640 1 Processor -- start 2024-02-21T23:25:54.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.910+0000 7f2bedf1e640 1 -- start start 2024-02-21T23:25:54.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.910+0000 7f59c5bd1640 1 -- 172.21.15.133:0/794784857 >> v2:172.21.15.133:3300/0 conn(0x7f59c014e510 msgr2=0x7f59c014e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.910+0000 7f59c5bd1640 1 --2- 172.21.15.133:0/794784857 >> v2:172.21.15.133:3300/0 conn(0x7f59c014e510 0x7f59c014e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.910+0000 7f59c5bd1640 1 -- 172.21.15.133:0/794784857 >> v1:172.21.15.142:6789/0 conn(0x7f59c00cdfe0 legacy=0x7f59c0070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.910+0000 7f59c5bd1640 1 -- 172.21.15.133:0/794784857 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00717e0 con 0x7f59c00ce3c0 2024-02-21T23:25:54.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.911+0000 7f2bedf1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2be80c3dc0 0x7f2be80ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.911+0000 7f2bedf1e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 0x7f2be8141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.911+0000 7f2bedf1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be8150d30 0x7f2be8148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.911+0000 7f2bedf1e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8070190 con 0x7f2be80c3dc0 2024-02-21T23:25:54.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.911+0000 7f2bedf1e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8157cd0 con 0x7f2be80c41a0 2024-02-21T23:25:54.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.911+0000 7f2bedf1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be815ad10 con 0x7f2be8150d30 2024-02-21T23:25:54.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.915+0000 7f091259c640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f0914151130 0x7f09140c0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:37416/0 (socket says 172.21.15.133:37416) 2024-02-21T23:25:54.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.915+0000 7f091259c640 1 -- 172.21.15.133:0/2960024446 learned_addr learned my addr 172.21.15.133:0/2960024446 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:54.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.919+0000 7f2e46ffd640 1 -- 172.21.15.133:0/320670964 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38034070 con 0x7f2e48154240 2024-02-21T23:25:54.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.919+0000 7f2e46ffd640 1 -- 172.21.15.133:0/320670964 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38030840 con 0x7f2e48154240 2024-02-21T23:25:54.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.919+0000 7f2e46ffd640 1 -- 172.21.15.133:0/320670964 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38030c10 con 0x7f2e48154240 2024-02-21T23:25:54.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.920+0000 7f2e4e51e640 1 -- 172.21.15.133:0/320670964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 msgr2=0x7f2e480c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.920+0000 7f2e4e51e640 1 --2- 172.21.15.133:0/320670964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 0x7f2e480c9bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009870 tx=0x7f2e3802b980 comp rx=0 tx=0).stop 2024-02-21T23:25:54.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.920+0000 7f2e4e51e640 1 -- 172.21.15.133:0/320670964 shutdown_connections 2024-02-21T23:25:54.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.920+0000 7f2e4e51e640 1 --2- 172.21.15.133:0/320670964 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4815d630 0x7f2e4815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.920+0000 7f2e4e51e640 1 --2- 172.21.15.133:0/320670964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 0x7f2e480c9bf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.920+0000 7f2e4e51e640 1 -- 172.21.15.133:0/320670964 >> 172.21.15.133:0/320670964 conn(0x7f2e4806f7f0 msgr2=0x7f2e48061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.920+0000 7f2e4e51e640 1 -- 172.21.15.133:0/320670964 shutdown_connections 2024-02-21T23:25:54.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.920+0000 7f2e4e51e640 1 -- 172.21.15.133:0/320670964 wait complete. 2024-02-21T23:25:54.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.921+0000 7f2e4e51e640 1 Processor -- start 2024-02-21T23:25:54.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.921+0000 7f2e4e51e640 1 -- start start 2024-02-21T23:25:54.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.922+0000 7f1e377fe640 1 -- 172.21.15.133:0/3513044872 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e3004a060 con 0x7f1e400c64b0 2024-02-21T23:25:54.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.922+0000 7f2e4e51e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e480c8420 0x7f2e481f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.922+0000 7f2e4e51e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 0x7f2e48154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.922+0000 7f1e377fe640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e2006b520 0x7f1e2006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.922+0000 7f2e4e51e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4815d630 0x7f2e481e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.922+0000 7f2e4e51e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480ca130 con 0x7f2e480c8420 2024-02-21T23:25:54.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.922+0000 7f2e4e51e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480c3c80 con 0x7f2e48154240 2024-02-21T23:25:54.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.922+0000 7f2e4e51e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480c36a0 con 0x7f2e4815d630 2024-02-21T23:25:54.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.922+0000 7f1e37fff640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e2006b520 0x7f1e2006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.923+0000 7f2be6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 0x7f2be8141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.923+0000 7f1e37fff640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e2006b520 0x7f1e2006d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e4014cca0 tx=0x7f1e28009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.923+0000 7f59c5bd1640 1 --2- 172.21.15.133:0/794784857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c00ce3c0 0x7f59c00c3250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f59c006b5c0 tx=0x7f59b002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e89c2e13290665 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.924+0000 7f2be6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 0x7f2be8141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42692/0 (socket says 172.21.15.133:42692) 2024-02-21T23:25:54.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.924+0000 7f2be6ffd640 1 -- 172.21.15.133:0/1489608373 learned_addr learned my addr 172.21.15.133:0/1489608373 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.926+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814935626 0 0) 0x7f0914072b00 con 0x7f0914151130 2024-02-21T23:25:54.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.926+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08fc003520 con 0x7f0914151130 2024-02-21T23:25:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.933+0000 7f2be6ffd640 1 -- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be8150d30 msgr2=0x7f2be8148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.933+0000 7f2be6ffd640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be8150d30 0x7f2be8148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.933+0000 7f2be6ffd640 1 -- 172.21.15.133:0/1489608373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2be80c3dc0 msgr2=0x7f2be80ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.933+0000 7f2be6ffd640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2be80c3dc0 0x7f2be80ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.933+0000 7f2be6ffd640 1 -- 172.21.15.133:0/1489608373 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80ca9b0 con 0x7f2be80c41a0 2024-02-21T23:25:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.933+0000 7f2e477fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 0x7f2e48154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.936+0000 7f2e477fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 0x7f2e48154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42694/0 (socket says 172.21.15.133:42694) 2024-02-21T23:25:54.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.936+0000 7f2e477fe640 1 -- 172.21.15.133:0/3303216338 learned_addr learned my addr 172.21.15.133:0/3303216338 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.937+0000 7f59c53d0640 1 -- 172.21.15.133:0/794784857 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0035020 con 0x7f59c00ce3c0 2024-02-21T23:25:54.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.937+0000 7f59c53d0640 1 -- 172.21.15.133:0/794784857 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b0031640 con 0x7f59c00ce3c0 2024-02-21T23:25:54.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.937+0000 7f59c53d0640 1 -- 172.21.15.133:0/794784857 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b002d040 con 0x7f59c00ce3c0 2024-02-21T23:25:54.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.937+0000 7f59c8636640 1 -- 172.21.15.133:0/794784857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c00ce3c0 msgr2=0x7f59c00c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.937+0000 7f59c8636640 1 --2- 172.21.15.133:0/794784857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c00ce3c0 0x7f59c00c3250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f59c006b5c0 tx=0x7f59b002b8f0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.938+0000 7f59c8636640 1 -- 172.21.15.133:0/794784857 shutdown_connections 2024-02-21T23:25:54.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.938+0000 7f59c8636640 1 --2- 172.21.15.133:0/794784857 >> v2:172.21.15.133:3300/0 conn(0x7f59c014e510 0x7f59c014e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.938+0000 7f59c8636640 1 --2- 172.21.15.133:0/794784857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c00ce3c0 0x7f59c00c3250 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.938+0000 7f59c8636640 1 -- 172.21.15.133:0/794784857 >> 172.21.15.133:0/794784857 conn(0x7f59c006aec0 msgr2=0x7f59c006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:54.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.938+0000 7f59c8636640 1 -- 172.21.15.133:0/794784857 shutdown_connections 2024-02-21T23:25:54.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.938+0000 7f59c8636640 1 -- 172.21.15.133:0/794784857 wait complete. 2024-02-21T23:25:54.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.939+0000 7f59c8636640 1 Processor -- start 2024-02-21T23:25:54.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.939+0000 7f59c8636640 1 -- start start 2024-02-21T23:25:54.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.940+0000 7f59c8636640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c00cdfe0 0x7f59c00b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.940+0000 7f59c8636640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59c00ce3c0 0x7f59c00b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.940+0000 7f59c8636640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c014e510 0x7f59c00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.940+0000 7f59c8636640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0158650 con 0x7f59c00ce3c0 2024-02-21T23:25:54.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.940+0000 7f59c8636640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0158370 con 0x7f59c00cdfe0 2024-02-21T23:25:54.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.940+0000 7f59c8636640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0158930 con 0x7f59c014e510 2024-02-21T23:25:54.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.941+0000 7f2be6ffd640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 0x7f2be8141a10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.943+0000 7f1e377fe640 1 -- 172.21.15.133:0/3513044872 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1e30078a20 con 0x7f1e400c64b0 2024-02-21T23:25:54.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.943+0000 7f1e377fe640 1 -- 172.21.15.133:0/3513044872 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f1e3003e070 con 0x7f1e400c64b0 2024-02-21T23:25:54.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.945+0000 7f2e477fe640 1 -- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4815d630 msgr2=0x7f2e481e4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.945+0000 7f2e477fe640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4815d630 0x7f2e481e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.945+0000 7f2e477fe640 1 -- 172.21.15.133:0/3303216338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e480c8420 msgr2=0x7f2e481f0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.945+0000 7f2e477fe640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e480c8420 0x7f2e481f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.945+0000 7f2e477fe640 1 -- 172.21.15.133:0/3303216338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4806d870 con 0x7f2e48154240 2024-02-21T23:25:54.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4176879227 0 0) 0x7f08fc003520 con 0x7f0914151130 2024-02-21T23:25:54.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 >> v2:172.21.15.133:3300/0 conn(0x7f0914072150 msgr2=0x7f09140725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 --2- 172.21.15.133:0/2960024446 >> v2:172.21.15.133:3300/0 conn(0x7f0914072150 0x7f09140725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 >> v1:172.21.15.133:6789/0 conn(0x7f0914156cd0 legacy=0x7f091406e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914157560 con 0x7f0914151130 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 797941756 0 0) 0x7f0904001670 con 0x7f0914151130 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 >> v1:172.21.15.142:6789/0 conn(0x7f0914151130 legacy=0x7f09140c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 --2- 172.21.15.133:0/2960024446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08fc003a00 0x7f08fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 --2- 172.21.15.133:0/2960024446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 --2- 172.21.15.133:0/2960024446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 0x7f08fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914157280 con 0x7f08fc004800 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc00c230 con 0x7f08fc003a00 2024-02-21T23:25:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.949+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc00c430 con 0x7f08fc0052e0 2024-02-21T23:25:54.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.952+0000 7f2e477fe640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 0x7f2e48154b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00c410 tx=0x7f2e3c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.953+0000 7f2bcffff640 1 -- 172.21.15.133:0/1489608373 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc03ad60 con 0x7f2be80c41a0 2024-02-21T23:25:54.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.953+0000 7f2bcffff640 1 -- 172.21.15.133:0/1489608373 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc0443f0 con 0x7f2be80c41a0 2024-02-21T23:25:54.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.953+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be81568a0 con 0x7f2be80c41a0 2024-02-21T23:25:54.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.953+0000 7f2bcffff640 1 -- 172.21.15.133:0/1489608373 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0447e0 con 0x7f2be80c41a0 2024-02-21T23:25:54.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.953+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be81ed9d0 con 0x7f2be80c41a0 2024-02-21T23:25:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.954+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be806a130 con 0x7f2be80c41a0 2024-02-21T23:25:54.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.958+0000 7f59c63d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c00cdfe0 0x7f59c00b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:54.979+0000 7fd1777fe640 1 -- 172.21.15.133:0/3739592566 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fd18404a360 con 0x7fd1901586d0 2024-02-21T23:25:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.000+0000 7f0912d9d640 1 --2- 172.21.15.133:0/2960024446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 0x7f08fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.002+0000 7f59c6bd3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c014e510 0x7f59c00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:54.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.002+0000 7f59c6bd3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c014e510 0x7f59c00b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45910/0 (socket says 172.21.15.133:45910) 2024-02-21T23:25:54.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.002+0000 7f59c6bd3640 1 -- 172.21.15.133:0/3612931746 learned_addr learned my addr 172.21.15.133:0/3612931746 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:54.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.005+0000 7f0912d9d640 1 -- 172.21.15.133:0/2960024446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08fc003a00 msgr2=0x7f08fc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.005+0000 7f0912d9d640 1 --2- 172.21.15.133:0/2960024446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08fc003a00 0x7f08fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.006+0000 7f0912d9d640 1 -- 172.21.15.133:0/2960024446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc004800 msgr2=0x7f08fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:54.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.006+0000 7f0912d9d640 1 --2- 172.21.15.133:0/2960024446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.006+0000 7f0912d9d640 1 -- 172.21.15.133:0/2960024446 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc003520 con 0x7f08fc0052e0 2024-02-21T23:25:54.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.007+0000 7f0912d9d640 1 -- 172.21.15.133:0/2960024446 reap_dead start 2024-02-21T23:25:54.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.009+0000 7f59c6bd3640 1 -- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c00cdfe0 msgr2=0x7f59c00b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.009+0000 7f59c6bd3640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c00cdfe0 0x7f59c00b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.009+0000 7f59c6bd3640 1 -- 172.21.15.133:0/3612931746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59c00ce3c0 msgr2=0x7f59c00b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:54.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.009+0000 7f59c6bd3640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59c00ce3c0 0x7f59c00b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:54.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.009+0000 7f59c6bd3640 1 -- 172.21.15.133:0/3612931746 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00b13c0 con 0x7f59c014e510 2024-02-21T23:25:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.012+0000 7f0912d9d640 1 --2- 172.21.15.133:0/2960024446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 0x7f08fc00b9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f090c009e10 tx=0x7f08fc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2005806a924e426e server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.012+0000 7f59c6bd3640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c014e510 0x7f59c00b15b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b970 tx=0x7f59b800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.019+0000 7f59b6ffd640 1 -- 172.21.15.133:0/3612931746 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8019070 con 0x7f59c014e510 2024-02-21T23:25:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.019+0000 7f59b6ffd640 1 -- 172.21.15.133:0/3612931746 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b800cb00 con 0x7f59c014e510 2024-02-21T23:25:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.019+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c00b29d0 con 0x7f59c014e510 2024-02-21T23:25:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.019+0000 7f59b6ffd640 1 -- 172.21.15.133:0/3612931746 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8014440 con 0x7f59c014e510 2024-02-21T23:25:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.019+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c01f0ce0 con 0x7f59c014e510 2024-02-21T23:25:55.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.019+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c009a10 con 0x7f08fc0052e0 2024-02-21T23:25:55.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.019+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c006e50 con 0x7f08fc0052e0 2024-02-21T23:25:55.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.019+0000 7f091159a640 1 -- 172.21.15.133:0/2960024446 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c003970 con 0x7f08fc0052e0 2024-02-21T23:25:55.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.020+0000 7f0918808640 1 -- 172.21.15.133:0/2960024446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 msgr2=0x7f08fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.020+0000 7f0918808640 1 --2- 172.21.15.133:0/2960024446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 0x7f08fc00b9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f090c009e10 tx=0x7f08fc005830 comp rx=0 tx=0).stop 2024-02-21T23:25:55.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.020+0000 7f0918808640 1 -- 172.21.15.133:0/2960024446 shutdown_connections 2024-02-21T23:25:55.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.020+0000 7f0918808640 1 --2- 172.21.15.133:0/2960024446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 0x7f08fc00b9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.020+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c006a130 con 0x7f59c014e510 2024-02-21T23:25:55.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.020+0000 7f0918808640 1 -- 172.21.15.133:0/2960024446 >> 172.21.15.133:0/2960024446 conn(0x7f091406f6e0 msgr2=0x7f091406faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:55.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.020+0000 7f0918808640 1 -- 172.21.15.133:0/2960024446 shutdown_connections 2024-02-21T23:25:55.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.020+0000 7f0918808640 1 -- 172.21.15.133:0/2960024446 wait complete. 2024-02-21T23:25:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.022+0000 7f0918808640 1 Processor -- start 2024-02-21T23:25:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.023+0000 7f0918808640 1 -- start start 2024-02-21T23:25:55.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.024+0000 7f0918808640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 0x7f09140c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.024+0000 7f0918808640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09141efb00 0x7f09140cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.024+0000 7f0918808640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091414f3a0 0x7f091414f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.024+0000 7f0918808640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914071950 con 0x7f09141efb00 2024-02-21T23:25:55.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.024+0000 7f0918808640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914157b20 con 0x7f091414f3a0 2024-02-21T23:25:55.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.024+0000 7f0918808640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0914071c30 con 0x7f08fc0052e0 2024-02-21T23:25:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.039+0000 7f59b6ffd640 1 -- 172.21.15.133:0/3612931746 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59b801c070 con 0x7f59c014e510 2024-02-21T23:25:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.040+0000 7f59b6ffd640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f59a406b4d0 0x7f59a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.040+0000 7f59c63d2640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f59a406b4d0 0x7f59a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:55.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.041+0000 7f59c63d2640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f59a406b4d0 0x7f59a406d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f59bc003d90 tx=0x7f59bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.092+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3fc8005850 con 0x7f3fe014ec20 2024-02-21T23:25:55.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.107+0000 7f4629ffb640 1 -- 172.21.15.133:0/1710479941 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462402cb10 con 0x7f462c0cd010 2024-02-21T23:25:55.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.107+0000 7f4629ffb640 1 -- 172.21.15.133:0/1710479941 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462402ccb0 con 0x7f462c0cd010 2024-02-21T23:25:55.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.107+0000 7f4629ffb640 1 -- 172.21.15.133:0/1710479941 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462402ce60 con 0x7f462c0cd010 2024-02-21T23:25:55.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.108+0000 7f46318d1640 1 -- 172.21.15.133:0/1710479941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 msgr2=0x7f462c15a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.108+0000 7f46318d1640 1 --2- 172.21.15.133:0/1710479941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 0x7f462c15a600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4624009e50 tx=0x7f462402bf70 comp rx=0 tx=0).stop 2024-02-21T23:25:55.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.108+0000 7f46318d1640 1 -- 172.21.15.133:0/1710479941 shutdown_connections 2024-02-21T23:25:55.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.108+0000 7f46318d1640 1 --2- 172.21.15.133:0/1710479941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 0x7f462c15a600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.108+0000 7f46318d1640 1 -- 172.21.15.133:0/1710479941 >> 172.21.15.133:0/1710479941 conn(0x7f462c061b50 msgr2=0x7f462c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:55.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.108+0000 7f46318d1640 1 -- 172.21.15.133:0/1710479941 shutdown_connections 2024-02-21T23:25:55.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.108+0000 7f46318d1640 1 -- 172.21.15.133:0/1710479941 wait complete. 2024-02-21T23:25:55.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.109+0000 7f46318d1640 1 Processor -- start 2024-02-21T23:25:55.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.110+0000 7f46318d1640 1 -- start start 2024-02-21T23:25:55.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.110+0000 7f46318d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f462c0c3dc0 0x7f462c1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.110+0000 7f46318d1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462c0c41a0 0x7f462c14c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.110+0000 7f46318d1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 0x7f462c14c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.110+0000 7f46318d1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c157cd0 con 0x7f462c0cd010 2024-02-21T23:25:55.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.110+0000 7f46318d1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c1579f0 con 0x7f462c0c3dc0 2024-02-21T23:25:55.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.110+0000 7f46318d1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c070190 con 0x7f462c0c41a0 2024-02-21T23:25:55.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.121+0000 7f0911d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09141efb00 0x7f09140cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:55.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.122+0000 7f0911d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09141efb00 0x7f09140cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40142/0 (socket says 172.21.15.133:40142) 2024-02-21T23:25:55.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.122+0000 7f0911d9b640 1 -- 172.21.15.133:0/3584046092 learned_addr learned my addr 172.21.15.133:0/3584046092 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:55.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.128+0000 7f0911d9b640 1 -- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 msgr2=0x7f09140c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.128+0000 7f0911d9b640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 0x7f09140c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.128+0000 7f0911d9b640 1 -- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091414f3a0 msgr2=0x7f091414f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.128+0000 7f0911d9b640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091414f3a0 0x7f091414f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.128+0000 7f0911d9b640 1 -- 172.21.15.133:0/3584046092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09140c70c0 con 0x7f09141efb00 2024-02-21T23:25:55.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.129+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1e40157f50 con 0x7f1e400c64b0 2024-02-21T23:25:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.147+0000 7f462b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 0x7f462c14c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:55.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.148+0000 7f0911d9b640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09141efb00 0x7f09140cb020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f090800e690 tx=0x7f090800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:55.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.149+0000 7f462b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 0x7f462c14c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40148/0 (socket says 172.21.15.133:40148) 2024-02-21T23:25:55.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.149+0000 7f462b7fe640 1 -- 172.21.15.133:0/1418693109 learned_addr learned my addr 172.21.15.133:0/1418693109 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.156+0000 7f0902ffd640 1 -- 172.21.15.133:0/3584046092 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908015070 con 0x7f09141efb00 2024-02-21T23:25:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.156+0000 7f0902ffd640 1 -- 172.21.15.133:0/3584046092 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0908005270 con 0x7f09141efb00 2024-02-21T23:25:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.156+0000 7f0902ffd640 1 -- 172.21.15.133:0/3584046092 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09080018c0 con 0x7f09141efb00 2024-02-21T23:25:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.156+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091414fd60 con 0x7f09141efb00 2024-02-21T23:25:55.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.157+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09140c7330 con 0x7f09141efb00 2024-02-21T23:25:55.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.158+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091406a130 con 0x7f09141efb00 2024-02-21T23:25:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.158+0000 7f1e377fe640 1 -- 172.21.15.133:0/3513044872 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e3007b080 con 0x7f1e400c64b0 2024-02-21T23:25:55.145 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-02-21T23:25:55.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.160+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e2006b520 msgr2=0x7f1e2006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.160+0000 7f1e456d5640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e2006b520 0x7f1e2006d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e4014cca0 tx=0x7f1e28009040 comp rx=0 tx=0).stop 2024-02-21T23:25:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e400c64b0 msgr2=0x7f1e400c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e400c64b0 0x7f1e400c3eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e30033000 tx=0x7f1e300046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 shutdown_connections 2024-02-21T23:25:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1e2006b520 0x7f1e2006d9e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e401588f0 0x7f1e40161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e400c7820 0x7f1e400ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 --2- 172.21.15.133:0/3513044872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e400c64b0 0x7f1e400c3eb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 >> 172.21.15.133:0/3513044872 conn(0x7f1e4006aec0 msgr2=0x7f1e40155fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 shutdown_connections 2024-02-21T23:25:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f1e456d5640 1 -- 172.21.15.133:0/3513044872 wait complete. 2024-02-21T23:25:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f462b7fe640 1 -- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462c0c41a0 msgr2=0x7f462c14c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f462b7fe640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462c0c41a0 0x7f462c14c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f462b7fe640 1 -- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f462c0c3dc0 msgr2=0x7f462c1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f462b7fe640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f462c0c3dc0 0x7f462c1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.161+0000 7f462b7fe640 1 -- 172.21.15.133:0/1418693109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c1f0410 con 0x7f462c0cd010 2024-02-21T23:25:55.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574860 for osd.0 2024-02-21T23:25:55.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.194+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd19015bfe0 con 0x7fd1901586d0 2024-02-21T23:25:55.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.250+0000 7f2bcffff640 1 -- 172.21.15.133:0/1489608373 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bdc04b060 con 0x7f2be80c41a0 2024-02-21T23:25:55.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.250+0000 7f2bcffff640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2bd406b550 0x7f2bd406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.250+0000 7f2be77fe640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2bd406b550 0x7f2bd406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:55.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.251+0000 7f2be77fe640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2bd406b550 0x7f2bd406da10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8001560 tx=0x7f2bd8007990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:55.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.252+0000 7f2e44ff9640 1 -- 172.21.15.133:0/3303216338 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c017070 con 0x7f2e48154240 2024-02-21T23:25:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.252+0000 7f2e44ff9640 1 -- 172.21.15.133:0/3303216338 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c0035f0 con 0x7f2e48154240 2024-02-21T23:25:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.252+0000 7f2e44ff9640 1 -- 172.21.15.133:0/3303216338 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0039e0 con 0x7f2e48154240 2024-02-21T23:25:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.252+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e48151570 con 0x7f2e48154240 2024-02-21T23:25:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.252+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e48151b30 con 0x7f2e48154240 2024-02-21T23:25:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.253+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4806a130 con 0x7f2e48154240 2024-02-21T23:25:55.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.258+0000 7f2bcffff640 1 -- 172.21.15.133:0/1489608373 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2bdc04ee40 con 0x7f2be80c41a0 2024-02-21T23:25:55.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.270+0000 7f2bcffff640 1 -- 172.21.15.133:0/1489608373 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f2be81568a0 con 0x7f2be80c41a0 2024-02-21T23:25:55.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.293+0000 7f59b6ffd640 1 -- 172.21.15.133:0/3612931746 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f59b8056c30 con 0x7f59c014e510 2024-02-21T23:25:55.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.311+0000 7f59b6ffd640 1 -- 172.21.15.133:0/3612931746 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f59b8023020 con 0x7f59c014e510 2024-02-21T23:25:55.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.320+0000 7fd1777fe640 1 -- 172.21.15.133:0/3739592566 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd18407ba50 con 0x7fd1901586d0 2024-02-21T23:25:55.305 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd17c06b4d0 msgr2=0x7fd17c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd17c06b4d0 0x7fd17c06d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd188001570 tx=0x7fd188007930 comp rx=0 tx=0).stop 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1901586d0 msgr2=0x7fd19015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1901586d0 0x7fd19015cbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd1840098c0 tx=0x7fd1840046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 shutdown_connections 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd17c06b4d0 0x7fd17c06d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1901586d0 0x7fd19015cbe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900c7f30 0x7fd19015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 --2- 172.21.15.133:0/3739592566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1900c65c0 0x7fd190161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 >> 172.21.15.133:0/3739592566 conn(0x7fd19006bac0 msgr2=0x7fd190162270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 shutdown_connections 2024-02-21T23:25:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.323+0000 7fd195399640 1 -- 172.21.15.133:0/3739592566 wait complete. 2024-02-21T23:25:55.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574860 for osd.1 2024-02-21T23:25:55.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.405+0000 7f462b7fe640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 0x7f462c14c620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4624036040 tx=0x7f4624004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:55.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.419+0000 7f0902ffd640 1 -- 172.21.15.133:0/3584046092 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0908018070 con 0x7f09141efb00 2024-02-21T23:25:55.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.420+0000 7f0902ffd640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f08f406b550 0x7f08f406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.420+0000 7f091259c640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f08f406b550 0x7f08f406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:55.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.420+0000 7f091259c640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f08f406b550 0x7f08f406da10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f09040079b0 tx=0x7f0904001600 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:55.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.443+0000 7f0902ffd640 1 -- 172.21.15.133:0/3584046092 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0908058ec0 con 0x7f09141efb00 2024-02-21T23:25:55.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.443+0000 7f0902ffd640 1 -- 172.21.15.133:0/3584046092 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f091414fd60 con 0x7f09141efb00 2024-02-21T23:25:55.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.452+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2be806b3d0 con 0x7f2be80c41a0 2024-02-21T23:25:55.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.457+0000 7f4613fff640 1 -- 172.21.15.133:0/1418693109 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624043020 con 0x7f462c0cd010 2024-02-21T23:25:55.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.457+0000 7f4613fff640 1 -- 172.21.15.133:0/1418693109 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462403e450 con 0x7f462c0cd010 2024-02-21T23:25:55.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.458+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c1f06d0 con 0x7f462c0cd010 2024-02-21T23:25:55.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.458+0000 7f4613fff640 1 -- 172.21.15.133:0/1418693109 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462402ce60 con 0x7f462c0cd010 2024-02-21T23:25:55.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.458+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c1598b0 con 0x7f462c0cd010 2024-02-21T23:25:55.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.459+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c06a130 con 0x7f462c0cd010 2024-02-21T23:25:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.484+0000 7f4613fff640 1 -- 172.21.15.133:0/1418693109 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4624046020 con 0x7f462c0cd010 2024-02-21T23:25:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.484+0000 7f4613fff640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f461806b4d0 0x7f461806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.485+0000 7f462affd640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f461806b4d0 0x7f461806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.485+0000 7f462affd640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f461806b4d0 0x7f461806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f461c009650 tx=0x7f461c006ba0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:55.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.535+0000 7f2bcffff640 1 -- 172.21.15.133:0/1489608373 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bdc03f070 con 0x7f2be80c41a0 2024-02-21T23:25:55.521 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-02-21T23:25:55.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2bd406b550 msgr2=0x7f2bd406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2bd406b550 0x7f2bd406da10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8001560 tx=0x7f2bd8007990 comp rx=0 tx=0).stop 2024-02-21T23:25:55.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 msgr2=0x7f2be8141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 0x7f2be8141a10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc0046c0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 shutdown_connections 2024-02-21T23:25:55.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2bd406b550 0x7f2bd406da10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be8150d30 0x7f2be8148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80c41a0 0x7f2be8141a10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 --2- 172.21.15.133:0/1489608373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2be80c3dc0 0x7f2be80ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 >> 172.21.15.133:0/1489608373 conn(0x7f2be8061b50 msgr2=0x7f2be814a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:55.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 shutdown_connections 2024-02-21T23:25:55.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.537+0000 7f2bedf1e640 1 -- 172.21.15.133:0/1489608373 wait complete. 2024-02-21T23:25:55.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574860 for osd.6 2024-02-21T23:25:55.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.549+0000 7f3fd67fc640 1 -- 172.21.15.133:0/1018913996 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3fd807f390 con 0x7f3fe014ec20 2024-02-21T23:25:55.535 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-02-21T23:25:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.549+0000 7f2e44ff9640 1 -- 172.21.15.133:0/3303216338 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e3c01a070 con 0x7f2e48154240 2024-02-21T23:25:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.549+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f59c01f0440 con 0x7f59c014e510 2024-02-21T23:25:55.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.550+0000 7f2e44ff9640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2e3406b500 0x7f2e3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:55.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.550+0000 7f2e47fff640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2e3406b500 0x7f2e3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:55.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.550+0000 7f2e47fff640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2e3406b500 0x7f2e3406d9c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e38036000 tx=0x7f2e3803b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:55.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3fc406b520 msgr2=0x7f3fc406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3fc406b520 0x7f3fc406d9e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0000f60 tx=0x7f3fd00079b0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe014ec20 msgr2=0x7f3fe01efa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe014ec20 0x7f3fe01efa30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009fa0 tx=0x7f3fd8004a40 comp rx=0 tx=0).stop 2024-02-21T23:25:55.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 shutdown_connections 2024-02-21T23:25:55.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3fc406b520 0x7f3fc406d9e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe014ec20 0x7f3fe01efa30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fe00ce3c0 0x7f3fe01ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 --2- 172.21.15.133:0/1018913996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe00cdfe0 0x7f3fe014b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 >> 172.21.15.133:0/1018913996 conn(0x7f3fe006aec0 msgr2=0x7f3fe015b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:55.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 shutdown_connections 2024-02-21T23:25:55.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.551+0000 7f3fe7ed5640 1 -- 172.21.15.133:0/1018913996 wait complete. 2024-02-21T23:25:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.554+0000 7f2e44ff9640 1 -- 172.21.15.133:0/3303216338 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2e3c056600 con 0x7f2e48154240 2024-02-21T23:25:55.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.555+0000 7f59b6ffd640 1 -- 172.21.15.133:0/3612931746 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f59b8059290 con 0x7f59c014e510 2024-02-21T23:25:55.540 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-02-21T23:25:55.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.557+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f59a406b4d0 msgr2=0x7f59a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.557+0000 7f59c8636640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f59a406b4d0 0x7f59a406d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f59bc003d90 tx=0x7f59bc009040 comp rx=0 tx=0).stop 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.557+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c014e510 msgr2=0x7f59c00b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.557+0000 7f59c8636640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c014e510 0x7f59c00b15b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b970 tx=0x7f59b800be30 comp rx=0 tx=0).stop 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.557+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 shutdown_connections 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.557+0000 7f59c8636640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f59a406b4d0 0x7f59a406d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.557+0000 7f59c8636640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c014e510 0x7f59c00b15b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.557+0000 7f59c8636640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59c00ce3c0 0x7f59c00b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.557+0000 7f59c8636640 1 --2- 172.21.15.133:0/3612931746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c00cdfe0 0x7f59c00b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.558+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 >> 172.21.15.133:0/3612931746 conn(0x7f59c006aec0 msgr2=0x7f59c00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.558+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 shutdown_connections 2024-02-21T23:25:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.558+0000 7f59c8636640 1 -- 172.21.15.133:0/3612931746 wait complete. 2024-02-21T23:25:55.549 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574860 for osd.7 2024-02-21T23:25:55.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.568+0000 7f2e44ff9640 1 -- 172.21.15.133:0/3303216338 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f2e3c021070 con 0x7f2e48154240 2024-02-21T23:25:55.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574860 for osd.2 2024-02-21T23:25:55.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.629+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f09140c7610 con 0x7f09141efb00 2024-02-21T23:25:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.749+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2e48062760 con 0x7f2e48154240 2024-02-21T23:25:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.752+0000 7f2e44ff9640 1 -- 172.21.15.133:0/3303216338 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e3c058c60 con 0x7f2e48154240 2024-02-21T23:25:55.738 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-02-21T23:25:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2e3406b500 msgr2=0x7f2e3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2e3406b500 0x7f2e3406d9c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e38036000 tx=0x7f2e3803b040 comp rx=0 tx=0).stop 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 msgr2=0x7f2e48154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 0x7f2e48154b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00c410 tx=0x7f2e3c00c8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 shutdown_connections 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f2e3406b500 0x7f2e3406d9c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4815d630 0x7f2e481e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e48154240 0x7f2e48154b70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 --2- 172.21.15.133:0/3303216338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e480c8420 0x7f2e481f0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 >> 172.21.15.133:0/3303216338 conn(0x7f2e4806f7f0 msgr2=0x7f2e4814a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 shutdown_connections 2024-02-21T23:25:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.755+0000 7f2e4e51e640 1 -- 172.21.15.133:0/3303216338 wait complete. 2024-02-21T23:25:55.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574860 for osd.4 2024-02-21T23:25:55.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.775+0000 7f4613fff640 1 -- 172.21.15.133:0/1418693109 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f46240795c0 con 0x7f462c0cd010 2024-02-21T23:25:55.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.775+0000 7f4613fff640 1 -- 172.21.15.133:0/1418693109 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f462404d030 con 0x7f462c0cd010 2024-02-21T23:25:55.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.785+0000 7f0902ffd640 1 -- 172.21.15.133:0/3584046092 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f090805b520 con 0x7f09141efb00 2024-02-21T23:25:55.770 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-02-21T23:25:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f08f406b550 msgr2=0x7f08f406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f08f406b550 0x7f08f406da10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f09040079b0 tx=0x7f0904001600 comp rx=0 tx=0).stop 2024-02-21T23:25:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09141efb00 msgr2=0x7f09140cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09141efb00 0x7f09140cb020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f090800e690 tx=0x7f090800eb50 comp rx=0 tx=0).stop 2024-02-21T23:25:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 shutdown_connections 2024-02-21T23:25:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f08f406b550 0x7f08f406da10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091414f3a0 0x7f091414f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09141efb00 0x7f09140cb020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 --2- 172.21.15.133:0/3584046092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08fc0052e0 0x7f09140c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 >> 172.21.15.133:0/3584046092 conn(0x7f091406f6e0 msgr2=0x7f091414b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 shutdown_connections 2024-02-21T23:25:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.787+0000 7f0918808640 1 -- 172.21.15.133:0/3584046092 wait complete. 2024-02-21T23:25:55.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574860 for osd.3 2024-02-21T23:25:55.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:55.969+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f462c1508e0 con 0x7f462c0cd010 2024-02-21T23:25:56.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.022+0000 7f4613fff640 1 -- 172.21.15.133:0/1418693109 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f462407bc20 con 0x7f462c0cd010 2024-02-21T23:25:56.008 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f461806b4d0 msgr2=0x7f461806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f461806b4d0 0x7f461806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f461c009650 tx=0x7f461c006ba0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 msgr2=0x7f462c14c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 0x7f462c14c620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4624036040 tx=0x7f4624004a40 comp rx=0 tx=0).stop 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 shutdown_connections 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f461806b4d0 0x7f461806d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462c0cd010 0x7f462c14c620 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462c0c41a0 0x7f462c14c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 --2- 172.21.15.133:0/1418693109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f462c0c3dc0 0x7f462c1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 >> 172.21.15.133:0/1418693109 conn(0x7f462c061b50 msgr2=0x7f462c0cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:56.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 shutdown_connections 2024-02-21T23:25:56.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.024+0000 7f46318d1640 1 -- 172.21.15.133:0/1418693109 wait complete. 2024-02-21T23:25:56.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.5 2024-02-21T23:25:56.021 DEBUG:teuthology.parallel:result is None 2024-02-21T23:25:56.159 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-02-21T23:25:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.276+0000 7f5d2eade640 1 Processor -- start 2024-02-21T23:25:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.276+0000 7f5d2eade640 1 -- start start 2024-02-21T23:25:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.277+0000 7f5d2eade640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5d280c53a0 0x7f5d28071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.277+0000 7f5d2eade640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 0x7f5d2815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.277+0000 7f5d2eade640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2815c120 con 0x7f5d280c4fc0 2024-02-21T23:25:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.277+0000 7f5d2eade640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d28158eb0 con 0x7f5d280c53a0 2024-02-21T23:25:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.277+0000 7f5d2eade640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28158230 con 0x7f5d2815b7b0 2024-02-21T23:25:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.277+0000 7f5d2d07b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 0x7f5d2815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.278+0000 7f5d27fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5d280c53a0 0x7f5d28071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.278+0000 7f5d2c87a640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f5d280c4fc0 0x7f5d2806f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:37432/0 (socket says 172.21.15.133:37432) 2024-02-21T23:25:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.278+0000 7f5d2c87a640 1 -- 172.21.15.133:0/3657538452 learned_addr learned my addr 172.21.15.133:0/3657538452 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.281+0000 7f5d2d07b640 1 -- 172.21.15.133:0/3657538452 >> v2:172.21.15.142:3300/0 conn(0x7f5d280c53a0 msgr2=0x7f5d28071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.281+0000 7f5d2d07b640 1 --2- 172.21.15.133:0/3657538452 >> v2:172.21.15.142:3300/0 conn(0x7f5d280c53a0 0x7f5d28071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.281+0000 7f5d2d07b640 1 -- 172.21.15.133:0/3657538452 >> v1:172.21.15.142:6789/0 conn(0x7f5d280c4fc0 legacy=0x7f5d2806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.281+0000 7f5d2d07b640 1 -- 172.21.15.133:0/3657538452 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d28158510 con 0x7f5d2815b7b0 2024-02-21T23:25:56.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.283+0000 7f5d2d07b640 1 --2- 172.21.15.133:0/3657538452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 0x7f5d2815bbe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009d90 tx=0x7f5d2002bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23924a00a4284d6e server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.288+0000 7f5d277fe640 1 -- 172.21.15.133:0/3657538452 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20039070 con 0x7f5d2815b7b0 2024-02-21T23:25:56.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.288+0000 7f5d277fe640 1 -- 172.21.15.133:0/3657538452 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2002cce0 con 0x7f5d2815b7b0 2024-02-21T23:25:56.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.288+0000 7f5d277fe640 1 -- 172.21.15.133:0/3657538452 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20034660 con 0x7f5d2815b7b0 2024-02-21T23:25:56.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.289+0000 7f5d2eade640 1 -- 172.21.15.133:0/3657538452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 msgr2=0x7f5d2815bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.289+0000 7f5d2eade640 1 --2- 172.21.15.133:0/3657538452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 0x7f5d2815bbe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009d90 tx=0x7f5d2002bf30 comp rx=0 tx=0).stop 2024-02-21T23:25:56.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.289+0000 7f5d2eade640 1 -- 172.21.15.133:0/3657538452 shutdown_connections 2024-02-21T23:25:56.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.289+0000 7f5d2eade640 1 --2- 172.21.15.133:0/3657538452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 0x7f5d2815bbe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.289+0000 7f5d2eade640 1 --2- 172.21.15.133:0/3657538452 >> v2:172.21.15.142:3300/0 conn(0x7f5d280c53a0 0x7f5d28071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.289+0000 7f5d2eade640 1 -- 172.21.15.133:0/3657538452 >> 172.21.15.133:0/3657538452 conn(0x7f5d2806aec0 msgr2=0x7f5d2806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:56.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.289+0000 7f5d2eade640 1 -- 172.21.15.133:0/3657538452 shutdown_connections 2024-02-21T23:25:56.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.289+0000 7f5d2eade640 1 -- 172.21.15.133:0/3657538452 wait complete. 2024-02-21T23:25:56.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.290+0000 7f5d2eade640 1 Processor -- start 2024-02-21T23:25:56.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.290+0000 7f5d2eade640 1 -- start start 2024-02-21T23:25:56.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.291+0000 7f5d2eade640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d280c4fc0 0x7f5d280cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.291+0000 7f5d2eade640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d280c53a0 0x7f5d280cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.291+0000 7f5d2eade640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 0x7f5d281f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.291+0000 7f5d2eade640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28157f20 con 0x7f5d280c4fc0 2024-02-21T23:25:56.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.291+0000 7f5d2eade640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28158ad0 con 0x7f5d280c53a0 2024-02-21T23:25:56.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.291+0000 7f5d2eade640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2806f8d0 con 0x7f5d2815b7b0 2024-02-21T23:25:56.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.291+0000 7f5d2c87a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d280c4fc0 0x7f5d280cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.292+0000 7f5d2c87a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d280c4fc0 0x7f5d280cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40158/0 (socket says 172.21.15.133:40158) 2024-02-21T23:25:56.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.292+0000 7f5d2c87a640 1 -- 172.21.15.133:0/2951711801 learned_addr learned my addr 172.21.15.133:0/2951711801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:56.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.292+0000 7f5d27fff640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d280c53a0 0x7f5d280cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.294+0000 7f5d2d07b640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 0x7f5d281f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.295+0000 7f5d2c87a640 1 -- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 msgr2=0x7f5d281f07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.295+0000 7f5d2c87a640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 0x7f5d281f07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.295+0000 7f5d2c87a640 1 -- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d280c53a0 msgr2=0x7f5d280cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.295+0000 7f5d2c87a640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d280c53a0 0x7f5d280cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.295+0000 7f5d2c87a640 1 -- 172.21.15.133:0/2951711801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d281edea0 con 0x7f5d280c4fc0 2024-02-21T23:25:56.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.298+0000 7f5d2c87a640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d280c4fc0 0x7f5d280cb550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c3f0 tx=0x7f5d1c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.301+0000 7f5d257fa640 1 -- 172.21.15.133:0/2951711801 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c017070 con 0x7f5d280c4fc0 2024-02-21T23:25:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.301+0000 7f5d257fa640 1 -- 172.21.15.133:0/2951711801 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c005ce0 con 0x7f5d280c4fc0 2024-02-21T23:25:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.301+0000 7f5d257fa640 1 -- 172.21.15.133:0/2951711801 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c007310 con 0x7f5d280c4fc0 2024-02-21T23:25:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.301+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d281ee160 con 0x7f5d280c4fc0 2024-02-21T23:25:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.301+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d281528b0 con 0x7f5d280c4fc0 2024-02-21T23:25:56.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.302+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2806a130 con 0x7f5d280c4fc0 2024-02-21T23:25:56.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.320+0000 7f5d257fa640 1 -- 172.21.15.133:0/2951711801 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d1c01a070 con 0x7f5d280c4fc0 2024-02-21T23:25:56.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.320+0000 7f5d257fa640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f5d0c06b520 0x7f5d0c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.321+0000 7f5d27fff640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f5d0c06b520 0x7f5d0c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.321+0000 7f5d27fff640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f5d0c06b520 0x7f5d0c06d9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d280cc600 tx=0x7f5d14009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.324+0000 7f5d257fa640 1 -- 172.21.15.133:0/2951711801 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f5d1c0561b0 con 0x7f5d280c4fc0 2024-02-21T23:25:56.321 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-02-21T23:25:56.348 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:25:56.366 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:25:56.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.438+0000 7f4da4b8c640 1 Processor -- start 2024-02-21T23:25:56.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.438+0000 7f4da4b8c640 1 -- start start 2024-02-21T23:25:56.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.439+0000 7f4da4b8c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4da0150f80 0x7f4da015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.439+0000 7f4da4b8c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4da015b020 0x7f4da00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.439+0000 7f4da4b8c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da00c2d70 con 0x7f4da00cbb30 2024-02-21T23:25:56.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.439+0000 7f4da4b8c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da00c2b40 con 0x7f4da0150f80 2024-02-21T23:25:56.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.439+0000 7f4da4b8c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da00c3050 con 0x7f4da015b020 2024-02-21T23:25:56.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.440+0000 7f4d9dd9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4da015b020 0x7f4da00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.440+0000 7f4d9dd9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4da015b020 0x7f4da00cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42764/0 (socket says 172.21.15.133:42764) 2024-02-21T23:25:56.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.440+0000 7f4d9dd9b640 1 -- 172.21.15.133:0/305986737 learned_addr learned my addr 172.21.15.133:0/305986737 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:56.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.443+0000 7f4d9dd9b640 1 -- 172.21.15.133:0/305986737 >> v2:172.21.15.133:3300/0 conn(0x7f4da0150f80 msgr2=0x7f4da015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:56.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.443+0000 7f4d9dd9b640 1 --2- 172.21.15.133:0/305986737 >> v2:172.21.15.133:3300/0 conn(0x7f4da0150f80 0x7f4da015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.443+0000 7f4d9dd9b640 1 -- 172.21.15.133:0/305986737 >> v1:172.21.15.133:6789/0 conn(0x7f4da00cbb30 legacy=0x7f4da00c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:56.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.443+0000 7f4d9dd9b640 1 -- 172.21.15.133:0/305986737 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00c3330 con 0x7f4da015b020 2024-02-21T23:25:56.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.445+0000 7f4d9dd9b640 1 --2- 172.21.15.133:0/305986737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da015b020 0x7f4da00cb5f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4da0071680 tx=0x7f4d9402bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd65adadc78185a server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.448+0000 7f4d9d59a640 1 -- 172.21.15.133:0/305986737 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9402cb20 con 0x7f4da015b020 2024-02-21T23:25:56.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.448+0000 7f4d9d59a640 1 -- 172.21.15.133:0/305986737 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9402ccc0 con 0x7f4da015b020 2024-02-21T23:25:56.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.448+0000 7f4d9d59a640 1 -- 172.21.15.133:0/305986737 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94034610 con 0x7f4da015b020 2024-02-21T23:25:56.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.449+0000 7f4da4b8c640 1 -- 172.21.15.133:0/305986737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da015b020 msgr2=0x7f4da00cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.449+0000 7f4da4b8c640 1 --2- 172.21.15.133:0/305986737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da015b020 0x7f4da00cb5f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4da0071680 tx=0x7f4d9402bf80 comp rx=0 tx=0).stop 2024-02-21T23:25:56.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.449+0000 7f4da4b8c640 1 -- 172.21.15.133:0/305986737 shutdown_connections 2024-02-21T23:25:56.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.449+0000 7f4da4b8c640 1 --2- 172.21.15.133:0/305986737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da015b020 0x7f4da00cb5f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.449+0000 7f4da4b8c640 1 --2- 172.21.15.133:0/305986737 >> v2:172.21.15.133:3300/0 conn(0x7f4da0150f80 0x7f4da015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.449+0000 7f4da4b8c640 1 -- 172.21.15.133:0/305986737 >> 172.21.15.133:0/305986737 conn(0x7f4da006aec0 msgr2=0x7f4da006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:56.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.449+0000 7f4da4b8c640 1 -- 172.21.15.133:0/305986737 shutdown_connections 2024-02-21T23:25:56.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.449+0000 7f4da4b8c640 1 -- 172.21.15.133:0/305986737 wait complete. 2024-02-21T23:25:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.450+0000 7f4da4b8c640 1 Processor -- start 2024-02-21T23:25:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.450+0000 7f4da4b8c640 1 -- start start 2024-02-21T23:25:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.451+0000 7f4da4b8c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da00cbb30 0x7f4da014ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.451+0000 7f4da4b8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4da0150f80 0x7f4da01e2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.451+0000 7f4da4b8c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da015b020 0x7f4da00c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.451+0000 7f4da4b8c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00c5990 con 0x7f4da015b020 2024-02-21T23:25:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.451+0000 7f4da4b8c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da015a5c0 con 0x7f4da00cbb30 2024-02-21T23:25:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.451+0000 7f4da4b8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00c5b10 con 0x7f4da0150f80 2024-02-21T23:25:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.451+0000 7f4d9e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da00cbb30 0x7f4da014ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.452+0000 7f4d9dd9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4da0150f80 0x7f4da01e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.452+0000 7f4d9e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da00cbb30 0x7f4da014ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42778/0 (socket says 172.21.15.133:42778) 2024-02-21T23:25:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.452+0000 7f4d9e59c640 1 -- 172.21.15.133:0/521165448 learned_addr learned my addr 172.21.15.133:0/521165448 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.455+0000 7f4d9e59c640 1 -- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4da0150f80 msgr2=0x7f4da01e2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.455+0000 7f4d9e59c640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4da0150f80 0x7f4da01e2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.455+0000 7f4d9e59c640 1 -- 172.21.15.133:0/521165448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da015b020 msgr2=0x7f4da00c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.455+0000 7f4d9e59c640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da015b020 0x7f4da00c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.455+0000 7f4d9e59c640 1 -- 172.21.15.133:0/521165448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da01ee060 con 0x7f4da00cbb30 2024-02-21T23:25:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.457+0000 7f4d9e59c640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da00cbb30 0x7f4da014ef80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800b4a0 tx=0x7f4d9800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.461+0000 7f4d8effd640 1 -- 172.21.15.133:0/521165448 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98014070 con 0x7f4da00cbb30 2024-02-21T23:25:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.461+0000 7f4d8effd640 1 -- 172.21.15.133:0/521165448 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98010610 con 0x7f4da00cbb30 2024-02-21T23:25:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.461+0000 7f4d8effd640 1 -- 172.21.15.133:0/521165448 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98010a00 con 0x7f4da00cbb30 2024-02-21T23:25:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.461+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da01e3e50 con 0x7f4da00cbb30 2024-02-21T23:25:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.461+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da01ee320 con 0x7f4da00cbb30 2024-02-21T23:25:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.461+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da0062400 con 0x7f4da00cbb30 2024-02-21T23:25:56.535 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-02-21T23:25:56.550 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-02-21T23:25:56.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.569+0000 7f5d257fa640 1 -- 172.21.15.133:0/2951711801 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f5d1c021020 con 0x7f5d280c4fc0 2024-02-21T23:25:56.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-02-21T23:25:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.656+0000 7f39c0b1e640 1 Processor -- start 2024-02-21T23:25:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.656+0000 7f39c0b1e640 1 -- start start 2024-02-21T23:25:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.657+0000 7f39c0b1e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f39bc0c5fd0 0x7f39bc06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.657+0000 7f39c0b1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc1587b0 0x7f39bc151960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.657+0000 7f39c0b1e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc070b40 con 0x7f39bc0c63b0 2024-02-21T23:25:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.657+0000 7f39c0b1e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc0cd460 con 0x7f39bc0c5fd0 2024-02-21T23:25:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.657+0000 7f39c0b1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0c7af0 con 0x7f39bc1587b0 2024-02-21T23:25:56.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.658+0000 7f39bad9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc1587b0 0x7f39bc151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.658+0000 7f39b3fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f39bc0c63b0 0x7f39bc1580a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51430/0 (socket says 172.21.15.133:51430) 2024-02-21T23:25:56.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.658+0000 7f39b3fff640 1 -- 172.21.15.133:0/391912134 learned_addr learned my addr 172.21.15.133:0/391912134 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:56.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.658+0000 7f39ba59c640 1 --2- 172.21.15.133:0/391912134 >> v2:172.21.15.133:3300/0 conn(0x7f39bc0c5fd0 0x7f39bc06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.662+0000 7f39ba59c640 1 -- 172.21.15.133:0/391912134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc1587b0 msgr2=0x7f39bc151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:56.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.662+0000 7f39ba59c640 1 --2- 172.21.15.133:0/391912134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc1587b0 0x7f39bc151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.662+0000 7f39ba59c640 1 -- 172.21.15.133:0/391912134 >> v1:172.21.15.133:6789/0 conn(0x7f39bc0c63b0 legacy=0x7f39bc1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.662+0000 7f39ba59c640 1 -- 172.21.15.133:0/391912134 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc070860 con 0x7f39bc0c5fd0 2024-02-21T23:25:56.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.670+0000 7f39ba59c640 1 --2- 172.21.15.133:0/391912134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc0c5fd0 0x7f39bc06b7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0097a0 tx=0x7f39ac02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74154e2e7557bc8 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.674+0000 7f39b9d9b640 1 -- 172.21.15.133:0/391912134 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac034070 con 0x7f39bc0c5fd0 2024-02-21T23:25:56.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.674+0000 7f39b9d9b640 1 -- 172.21.15.133:0/391912134 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac030800 con 0x7f39bc0c5fd0 2024-02-21T23:25:56.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.674+0000 7f39b9d9b640 1 -- 172.21.15.133:0/391912134 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac030bd0 con 0x7f39bc0c5fd0 2024-02-21T23:25:56.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7f39c0b1e640 1 -- 172.21.15.133:0/391912134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc0c5fd0 msgr2=0x7f39bc06b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7f39c0b1e640 1 --2- 172.21.15.133:0/391912134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc0c5fd0 0x7f39bc06b7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0097a0 tx=0x7f39ac02b940 comp rx=0 tx=0).stop 2024-02-21T23:25:56.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7f39c0b1e640 1 -- 172.21.15.133:0/391912134 shutdown_connections 2024-02-21T23:25:56.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7f39c0b1e640 1 --2- 172.21.15.133:0/391912134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc1587b0 0x7f39bc151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7f39c0b1e640 1 --2- 172.21.15.133:0/391912134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc0c5fd0 0x7f39bc06b7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7f39c0b1e640 1 -- 172.21.15.133:0/391912134 >> 172.21.15.133:0/391912134 conn(0x7f39bc061b50 msgr2=0x7f39bc072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:56.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7f39c0b1e640 1 -- 172.21.15.133:0/391912134 shutdown_connections 2024-02-21T23:25:56.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7f39c0b1e640 1 -- 172.21.15.133:0/391912134 wait complete. 2024-02-21T23:25:56.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7fd9753fa640 1 Processor -- start 2024-02-21T23:25:56.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.675+0000 7fd9753fa640 1 -- start start 2024-02-21T23:25:56.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.676+0000 7fd9753fa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd9701542a0 0x7fd97015e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.676+0000 7fd9753fa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd970072fe0 0x7fd9700723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.676+0000 7fd9753fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072900 0x7fd9700c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.676+0000 7fd9753fa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97015ade0 con 0x7fd9701542a0 2024-02-21T23:25:56.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.676+0000 7fd9753fa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97015e5b0 con 0x7fd970072fe0 2024-02-21T23:25:56.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.676+0000 7fd9753fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd970071c60 con 0x7fd970072900 2024-02-21T23:25:56.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.677+0000 7f39c0b1e640 1 Processor -- start 2024-02-21T23:25:56.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.677+0000 7f39c0b1e640 1 -- start start 2024-02-21T23:25:56.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.677+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072900 0x7fd9700c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.677+0000 7fd96effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd9701542a0 0x7fd97015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.677+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072900 0x7fd9700c1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:45994/0 (socket says 172.21.15.133:45994) 2024-02-21T23:25:56.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.677+0000 7fd96f7fe640 1 -- 172.21.15.133:0/2879673905 learned_addr learned my addr 172.21.15.133:0/2879673905 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:56.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.678+0000 7f39c0b1e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc0c5fd0 0x7f39bc1f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.678+0000 7f39c0b1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc0c63b0 0x7f39bc1f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.678+0000 7f39c0b1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1587b0 0x7f39bc1ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.678+0000 7f39c0b1e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0cd0f0 con 0x7f39bc1587b0 2024-02-21T23:25:56.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.678+0000 7f39c0b1e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0cce10 con 0x7f39bc0c5fd0 2024-02-21T23:25:56.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.678+0000 7f39c0b1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc06bce0 con 0x7f39bc0c63b0 2024-02-21T23:25:56.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.680+0000 7fd96f7fe640 1 -- 172.21.15.133:0/2879673905 >> v2:172.21.15.142:3300/0 conn(0x7fd970072fe0 msgr2=0x7fd9700723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:56.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.680+0000 7fd96f7fe640 1 --2- 172.21.15.133:0/2879673905 >> v2:172.21.15.142:3300/0 conn(0x7fd970072fe0 0x7fd9700723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.680+0000 7fd96f7fe640 1 -- 172.21.15.133:0/2879673905 >> v2:172.21.15.133:3300/0 conn(0x7fd9701542a0 msgr2=0x7fd97015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:56.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.680+0000 7fd96f7fe640 1 --2- 172.21.15.133:0/2879673905 >> v2:172.21.15.133:3300/0 conn(0x7fd9701542a0 0x7fd97015e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.680+0000 7fd96f7fe640 1 -- 172.21.15.133:0/2879673905 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd970071f00 con 0x7fd970072900 2024-02-21T23:25:56.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.683+0000 7f39b3fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc0c63b0 0x7f39bc1f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.683+0000 7fd96f7fe640 1 --2- 172.21.15.133:0/2879673905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072900 0x7fd9700c1bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd968009e50 tx=0x7fd96802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aba9f1d715e3aa6c server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.684+0000 7f39bad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1587b0 0x7f39bc1ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.684+0000 7f39b3fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc0c63b0 0x7f39bc1f1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:46008/0 (socket says 172.21.15.133:46008) 2024-02-21T23:25:56.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.684+0000 7f39b3fff640 1 -- 172.21.15.133:0/1609713176 learned_addr learned my addr 172.21.15.133:0/1609713176 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:56.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.689+0000 7f39bad9d640 1 -- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc0c63b0 msgr2=0x7f39bc1f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.689+0000 7f39bad9d640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc0c63b0 0x7f39bc1f1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.689+0000 7f39bad9d640 1 -- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc0c5fd0 msgr2=0x7f39bc1f0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.689+0000 7f39bad9d640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc0c5fd0 0x7f39bc1f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.689+0000 7f39bad9d640 1 -- 172.21.15.133:0/1609713176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc153780 con 0x7f39bc1587b0 2024-02-21T23:25:56.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.692+0000 7f39bad9d640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1587b0 0x7f39bc1ef3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39b400eba0 tx=0x7f39b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.693+0000 7fd96dffb640 1 -- 172.21.15.133:0/2879673905 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96802cb00 con 0x7fd970072900 2024-02-21T23:25:56.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.693+0000 7fd96dffb640 1 -- 172.21.15.133:0/2879673905 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96802cca0 con 0x7fd970072900 2024-02-21T23:25:56.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.693+0000 7fd96dffb640 1 -- 172.21.15.133:0/2879673905 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968034610 con 0x7fd970072900 2024-02-21T23:25:56.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.694+0000 7fd9753fa640 1 -- 172.21.15.133:0/2879673905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072900 msgr2=0x7fd9700c1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.694+0000 7fd9753fa640 1 --2- 172.21.15.133:0/2879673905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072900 0x7fd9700c1bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd968009e50 tx=0x7fd96802bf60 comp rx=0 tx=0).stop 2024-02-21T23:25:56.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.694+0000 7fd9753fa640 1 -- 172.21.15.133:0/2879673905 shutdown_connections 2024-02-21T23:25:56.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.694+0000 7fd9753fa640 1 --2- 172.21.15.133:0/2879673905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072900 0x7fd9700c1bb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.694+0000 7fd9753fa640 1 --2- 172.21.15.133:0/2879673905 >> v2:172.21.15.142:3300/0 conn(0x7fd970072fe0 0x7fd9700723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.694+0000 7fd9753fa640 1 --2- 172.21.15.133:0/2879673905 >> v2:172.21.15.133:3300/0 conn(0x7fd9701542a0 0x7fd97015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.694+0000 7fd9753fa640 1 -- 172.21.15.133:0/2879673905 >> 172.21.15.133:0/2879673905 conn(0x7fd97006f6e0 msgr2=0x7fd97006faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:56.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.694+0000 7fd9753fa640 1 -- 172.21.15.133:0/2879673905 shutdown_connections 2024-02-21T23:25:56.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.694+0000 7fd9753fa640 1 -- 172.21.15.133:0/2879673905 wait complete. 2024-02-21T23:25:56.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.696+0000 7fd9753fa640 1 Processor -- start 2024-02-21T23:25:56.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.697+0000 7fd9753fa640 1 -- start start 2024-02-21T23:25:56.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.698+0000 7fd9753fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd970072900 0x7fd9700c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.698+0000 7fd9753fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072fe0 0x7fd9701f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.698+0000 7fd9753fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9701542a0 0x7fd9701f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.698+0000 7fd9753fa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97015a790 con 0x7fd970072900 2024-02-21T23:25:56.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.698+0000 7fd9753fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97015a4b0 con 0x7fd9701542a0 2024-02-21T23:25:56.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.698+0000 7fd9753fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97015aa70 con 0x7fd970072fe0 2024-02-21T23:25:56.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.698+0000 7fd96e7fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072fe0 0x7fd9701f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.699+0000 7fd96e7fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072fe0 0x7fd9701f0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:46012/0 (socket says 172.21.15.133:46012) 2024-02-21T23:25:56.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.699+0000 7fd96e7fc640 1 -- 172.21.15.133:0/1153925734 learned_addr learned my addr 172.21.15.133:0/1153925734 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:56.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.704+0000 7fd96e7fc640 1 -- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9701542a0 msgr2=0x7fd9701f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.704+0000 7fd96e7fc640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9701542a0 0x7fd9701f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.704+0000 7fd96e7fc640 1 -- 172.21.15.133:0/1153925734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd970072900 msgr2=0x7fd9700c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.704+0000 7fd96e7fc640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd970072900 0x7fd9700c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.704+0000 7fd96e7fc640 1 -- 172.21.15.133:0/1153925734 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9701f0d80 con 0x7fd970072fe0 2024-02-21T23:25:56.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.707+0000 7fd96e7fc640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072fe0 0x7fd9701f0840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd96400b4d0 tx=0x7fd96400b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.710+0000 7fd9577fe640 1 -- 172.21.15.133:0/1153925734 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964015020 con 0x7fd970072fe0 2024-02-21T23:25:56.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.710+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97015d080 con 0x7fd970072fe0 2024-02-21T23:25:56.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.710+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97015d350 con 0x7fd970072fe0 2024-02-21T23:25:56.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.711+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97006efc0 con 0x7fd970072fe0 2024-02-21T23:25:56.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.713+0000 7fd9577fe640 1 -- 172.21.15.133:0/1153925734 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd964011890 con 0x7fd970072fe0 2024-02-21T23:25:56.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.713+0000 7fd9577fe640 1 -- 172.21.15.133:0/1153925734 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9640092b0 con 0x7fd970072fe0 2024-02-21T23:25:56.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.715+0000 7f39b2ffd640 1 -- 172.21.15.133:0/1609713176 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4018020 con 0x7f39bc1587b0 2024-02-21T23:25:56.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.715+0000 7f39b2ffd640 1 -- 172.21.15.133:0/1609713176 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b400ccc0 con 0x7f39bc1587b0 2024-02-21T23:25:56.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.715+0000 7f39b2ffd640 1 -- 172.21.15.133:0/1609713176 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b40146c0 con 0x7f39bc1587b0 2024-02-21T23:25:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.715+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc1531c0 con 0x7f39bc1587b0 2024-02-21T23:25:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.715+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc1eebf0 con 0x7f39bc1587b0 2024-02-21T23:25:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.716+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc06a130 con 0x7f39bc1587b0 2024-02-21T23:25:56.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.717+0000 7f4d8effd640 1 -- 172.21.15.133:0/521165448 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d98017070 con 0x7f4da00cbb30 2024-02-21T23:25:56.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.717+0000 7f4d8effd640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4d8006b4d0 0x7f4d8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.717+0000 7f4d9dd9b640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4d8006b4d0 0x7f4d8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.718+0000 7f4d9dd9b640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4d8006b4d0 0x7f4d8006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d94035000 tx=0x7f4d94030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.720+0000 7f4d8effd640 1 -- 172.21.15.133:0/521165448 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f4d98056940 con 0x7f4da00cbb30 2024-02-21T23:25:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.725+0000 7f829f436640 1 Processor -- start 2024-02-21T23:25:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.725+0000 7f829f436640 1 -- start start 2024-02-21T23:25:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.726+0000 7f829f436640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82980cfe10 0x7f82980d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.726+0000 7f829f436640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82980cc890 0x7f82980ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.726+0000 7f829f436640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82980cc100 con 0x7f829815a170 2024-02-21T23:25:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.726+0000 7f829f436640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82980cc400 con 0x7f82980cc890 2024-02-21T23:25:56.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.726+0000 7f829f436640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829815a6b0 con 0x7f82980cfe10 2024-02-21T23:25:56.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.730+0000 7f4d8effd640 1 -- 172.21.15.133:0/521165448 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f4da0062400 con 0x7f4da00cbb30 2024-02-21T23:25:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.733+0000 7f829d9d3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82980cc890 0x7f82980ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.735+0000 7f829d9d3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82980cc890 0x7f82980ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42808/0 (socket says 172.21.15.133:42808) 2024-02-21T23:25:56.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.735+0000 7f829d9d3640 1 -- 172.21.15.133:0/851110822 learned_addr learned my addr 172.21.15.133:0/851110822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:56.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.738+0000 7f829d9d3640 1 -- 172.21.15.133:0/851110822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82980cfe10 msgr2=0x7f82980d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:56.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.738+0000 7f829d9d3640 1 --2- 172.21.15.133:0/851110822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82980cfe10 0x7f82980d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.738+0000 7f829d9d3640 1 -- 172.21.15.133:0/851110822 >> v1:172.21.15.133:6789/0 conn(0x7f829815a170 legacy=0x7f82980ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:56.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.738+0000 7f829d9d3640 1 -- 172.21.15.133:0/851110822 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82980cd260 con 0x7f82980cc890 2024-02-21T23:25:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.740+0000 7f829d9d3640 1 --2- 172.21.15.133:0/851110822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cc890 0x7f82980ccd20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8290009870 tx=0x7f829002b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2149212e0b56ae2 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.742+0000 7f828ffff640 1 -- 172.21.15.133:0/851110822 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900317c0 con 0x7f82980cc890 2024-02-21T23:25:56.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.742+0000 7f39b2ffd640 1 -- 172.21.15.133:0/1609713176 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39b4005d10 con 0x7f39bc1587b0 2024-02-21T23:25:56.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.743+0000 7f39b2ffd640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f399c06b550 0x7f399c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.743+0000 7f39ba59c640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f399c06b550 0x7f399c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.744+0000 7f39ba59c640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f399c06b550 0x7f399c06da10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39ac036000 tx=0x7f39ac030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.744+0000 7f828ffff640 1 -- 172.21.15.133:0/851110822 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290037070 con 0x7f82980cc890 2024-02-21T23:25:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.744+0000 7f828ffff640 1 -- 172.21.15.133:0/851110822 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290031b80 con 0x7f82980cc890 2024-02-21T23:25:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.745+0000 7f829f436640 1 -- 172.21.15.133:0/851110822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cc890 msgr2=0x7f82980ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.745+0000 7f829f436640 1 --2- 172.21.15.133:0/851110822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cc890 0x7f82980ccd20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8290009870 tx=0x7f829002b860 comp rx=0 tx=0).stop 2024-02-21T23:25:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.745+0000 7f829f436640 1 -- 172.21.15.133:0/851110822 shutdown_connections 2024-02-21T23:25:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.745+0000 7f829f436640 1 --2- 172.21.15.133:0/851110822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cc890 0x7f82980ccd20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.745+0000 7f829f436640 1 --2- 172.21.15.133:0/851110822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82980cfe10 0x7f82980d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.745+0000 7f829f436640 1 -- 172.21.15.133:0/851110822 >> 172.21.15.133:0/851110822 conn(0x7f8298061b50 msgr2=0x7f8298061f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.745+0000 7f829f436640 1 -- 172.21.15.133:0/851110822 shutdown_connections 2024-02-21T23:25:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.745+0000 7f829f436640 1 -- 172.21.15.133:0/851110822 wait complete. 2024-02-21T23:25:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.747+0000 7f829f436640 1 Processor -- start 2024-02-21T23:25:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.747+0000 7f829f436640 1 -- start start 2024-02-21T23:25:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.748+0000 7f829f436640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82980cc890 0x7f829814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.748+0000 7f829f436640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cfe10 0x7f82981f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.748+0000 7f829f436640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f829815a170 0x7f82980c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.748+0000 7f829f436640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829815af30 con 0x7f829815a170 2024-02-21T23:25:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.748+0000 7f829f436640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829815ac50 con 0x7f82980cfe10 2024-02-21T23:25:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.748+0000 7f829f436640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980cab50 con 0x7f82980cc890 2024-02-21T23:25:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.749+0000 7f829c9d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cfe10 0x7f82981f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.749+0000 7f829c9d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cfe10 0x7f82981f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42816/0 (socket says 172.21.15.133:42816) 2024-02-21T23:25:56.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.749+0000 7f829c9d1640 1 -- 172.21.15.133:0/2845279153 learned_addr learned my addr 172.21.15.133:0/2845279153 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:56.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.752+0000 7f829c9d1640 1 -- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82980cc890 msgr2=0x7f829814c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.752+0000 7f829c9d1640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82980cc890 0x7f829814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.752+0000 7f829c9d1640 1 -- 172.21.15.133:0/2845279153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f829815a170 msgr2=0x7f82980c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:56.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.752+0000 7f829c9d1640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f829815a170 0x7f82980c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:56.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.752+0000 7f829c9d1640 1 -- 172.21.15.133:0/2845279153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82980c3840 con 0x7f82980cfe10 2024-02-21T23:25:56.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.754+0000 7f829c9d1640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cfe10 0x7f82981f3130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f828800b3e0 tx=0x7f828800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.758+0000 7f828dffb640 1 -- 172.21.15.133:0/2845279153 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288015020 con 0x7f82980cfe10 2024-02-21T23:25:56.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.758+0000 7f828dffb640 1 -- 172.21.15.133:0/2845279153 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8288011850 con 0x7f82980cfe10 2024-02-21T23:25:56.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.758+0000 7f828dffb640 1 -- 172.21.15.133:0/2845279153 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288011c40 con 0x7f82980cfe10 2024-02-21T23:25:56.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.758+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82980c3b20 con 0x7f82980cfe10 2024-02-21T23:25:56.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.759+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8298152f20 con 0x7f82980cfe10 2024-02-21T23:25:56.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.760+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829806a130 con 0x7f82980cfe10 2024-02-21T23:25:56.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.761+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5d281520c0 con 0x7f5d280c4fc0 2024-02-21T23:25:56.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-02-21T23:25:56.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.784+0000 7f828dffb640 1 -- 172.21.15.133:0/2845279153 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8288018020 con 0x7f82980cfe10 2024-02-21T23:25:56.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.784+0000 7f828dffb640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f827c06b4d0 0x7f827c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.785+0000 7f829d1d2640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f827c06b4d0 0x7f827c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.785+0000 7f829d1d2640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f827c06b4d0 0x7f827c06d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f829400ab90 tx=0x7f8294009250 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:56.785 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-02-21T23:25:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.879+0000 7faa464a2640 1 Processor -- start 2024-02-21T23:25:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.879+0000 7faa464a2640 1 -- start start 2024-02-21T23:25:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.880+0000 7faa464a2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faa40154240 0x7faa400ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.880+0000 7faa464a2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa400703c0 0x7faa400c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.880+0000 7faa464a2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa400cd300 con 0x7faa40070800 2024-02-21T23:25:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.880+0000 7faa464a2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa400cd570 con 0x7faa400703c0 2024-02-21T23:25:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.880+0000 7faa464a2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4015a130 con 0x7faa40154240 2024-02-21T23:25:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.903+0000 7f15d971e640 1 Processor -- start 2024-02-21T23:25:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.903+0000 7f15d971e640 1 -- start start 2024-02-21T23:25:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.904+0000 7f15d971e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f15d40c6f50 0x7f15d40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.904+0000 7f15d971e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d415ab40 con 0x7f15d4154600 2024-02-21T23:25:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.904+0000 7f15d971e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d415ae00 con 0x7f15d40c7c10 2024-02-21T23:25:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.904+0000 7f15d971e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d40c74a0 con 0x7f15d40c6f50 2024-02-21T23:25:56.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.919+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4da0071a00 con 0x7f4da00cbb30 2024-02-21T23:25:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.995+0000 7f39b2ffd640 1 -- 172.21.15.133:0/1609713176 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f39b4057b10 con 0x7f39bc1587b0 2024-02-21T23:25:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:56.995+0000 7f39b2ffd640 1 -- 172.21.15.133:0/1609713176 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f39b4022030 con 0x7f39bc1587b0 2024-02-21T23:25:56.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.008+0000 7fd9577fe640 1 -- 172.21.15.133:0/1153925734 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd964018020 con 0x7fd970072fe0 2024-02-21T23:25:56.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.008+0000 7fd9577fe640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd95c06b4d0 0x7fd95c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:56.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.009+0000 7fd96effd640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd95c06b4d0 0x7fd95c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:56.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.009+0000 7fd96effd640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd95c06b4d0 0x7fd95c06d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd960006a90 tx=0x7fd960006a20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:57.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.014+0000 7fd9577fe640 1 -- 172.21.15.133:0/1153925734 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fd964057140 con 0x7fd970072fe0 2024-02-21T23:25:57.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.030+0000 7fd9577fe640 1 -- 172.21.15.133:0/1153925734 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fd97006efc0 con 0x7fd970072fe0 2024-02-21T23:25:57.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.049+0000 7f828dffb640 1 -- 172.21.15.133:0/2845279153 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f8288011de0 con 0x7f82980cfe10 2024-02-21T23:25:57.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.049+0000 7f5d257fa640 1 -- 172.21.15.133:0/2951711801 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d1c058110 con 0x7f5d280c4fc0 2024-02-21T23:25:57.035 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:57.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.051+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f5d0c06b520 msgr2=0x7f5d0c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.051+0000 7f5d2eade640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f5d0c06b520 0x7f5d0c06d9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d280cc600 tx=0x7f5d14009190 comp rx=0 tx=0).stop 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.051+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d280c4fc0 msgr2=0x7f5d280cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.051+0000 7f5d2eade640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d280c4fc0 0x7f5d280cb550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c3f0 tx=0x7f5d1c00c8b0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.052+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 shutdown_connections 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.052+0000 7f5d2eade640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f5d0c06b520 0x7f5d0c06d9e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.052+0000 7f5d2eade640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d2815b7b0 0x7f5d281f07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.052+0000 7f5d2eade640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d280c53a0 0x7f5d280cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.052+0000 7f5d2eade640 1 --2- 172.21.15.133:0/2951711801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d280c4fc0 0x7f5d280cb550 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.052+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 >> 172.21.15.133:0/2951711801 conn(0x7f5d2806aec0 msgr2=0x7f5d2806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.052+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 shutdown_connections 2024-02-21T23:25:57.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.052+0000 7f5d2eade640 1 -- 172.21.15.133:0/2951711801 wait complete. 2024-02-21T23:25:57.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.055+0000 7faa3f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa400703c0 0x7faa400c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.057+0000 7faa3f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa400703c0 0x7faa400c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40220/0 (socket says 172.21.15.133:40220) 2024-02-21T23:25:57.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.057+0000 7faa3f7fe640 1 -- 172.21.15.133:0/4179646011 learned_addr learned my addr 172.21.15.133:0/4179646011 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:57.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.060+0000 7faa3f7fe640 1 -- 172.21.15.133:0/4179646011 >> v2:172.21.15.142:3300/0 conn(0x7faa40154240 msgr2=0x7faa400ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:57.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.060+0000 7faa3f7fe640 1 --2- 172.21.15.133:0/4179646011 >> v2:172.21.15.142:3300/0 conn(0x7faa40154240 0x7faa400ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.060+0000 7faa3f7fe640 1 -- 172.21.15.133:0/4179646011 >> v1:172.21.15.142:6789/0 conn(0x7faa40070800 legacy=0x7faa4015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:57.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.060+0000 7faa3f7fe640 1 -- 172.21.15.133:0/4179646011 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4015a410 con 0x7faa400703c0 2024-02-21T23:25:57.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.062+0000 7f828dffb640 1 -- 172.21.15.133:0/2845279153 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f828801f020 con 0x7f82980cfe10 2024-02-21T23:25:57.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.064+0000 7f15d37fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f15d40c7c10 0x7f15d40c8060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:37450/0 (socket says 172.21.15.133:37450) 2024-02-21T23:25:57.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.064+0000 7f15d37fe640 1 -- 172.21.15.133:0/3246123184 learned_addr learned my addr 172.21.15.133:0/3246123184 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:57.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.0 2024-02-21T23:25:57.051 DEBUG:teuthology.parallel:result is None 2024-02-21T23:25:57.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.066+0000 7f15d27fc640 1 --2- 172.21.15.133:0/3246123184 >> v2:172.21.15.133:3300/0 conn(0x7f15d40c6f50 0x7f15d40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.068+0000 7faa3f7fe640 1 --2- 172.21.15.133:0/4179646011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa400703c0 0x7faa400c90b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa40072a30 tx=0x7faa3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63aa762d09c1203 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:57.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.074+0000 7f15d27fc640 1 -- 172.21.15.133:0/3246123184 >> v1:172.21.15.142:6789/0 conn(0x7f15d40c7c10 legacy=0x7f15d40c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.074+0000 7f15d27fc640 1 -- 172.21.15.133:0/3246123184 >> v1:172.21.15.133:6789/0 conn(0x7f15d4154600 legacy=0x7f15d40ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:57.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.074+0000 7f15d27fc640 1 -- 172.21.15.133:0/3246123184 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40cb580 con 0x7f15d40c6f50 2024-02-21T23:25:57.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.076+0000 7f4d8effd640 1 -- 172.21.15.133:0/521165448 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4da0071a00 con 0x7f4da00cbb30 2024-02-21T23:25:57.062 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:57.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f15d27fc640 1 --2- 172.21.15.133:0/3246123184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d40c6f50 0x7f15d40cbc00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15c80097a0 tx=0x7f15c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a00d50d31443d33 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:57.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4d8006b4d0 msgr2=0x7f4d8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4d8006b4d0 0x7f4d8006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d94035000 tx=0x7f4d94030000 comp rx=0 tx=0).stop 2024-02-21T23:25:57.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da00cbb30 msgr2=0x7f4da014ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da00cbb30 0x7f4da014ef80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800b4a0 tx=0x7f4d9800b960 comp rx=0 tx=0).stop 2024-02-21T23:25:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 shutdown_connections 2024-02-21T23:25:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4d8006b4d0 0x7f4d8006d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da015b020 0x7f4da00c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4da0150f80 0x7f4da01e2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 --2- 172.21.15.133:0/521165448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4da00cbb30 0x7f4da014ef80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 >> 172.21.15.133:0/521165448 conn(0x7f4da006aec0 msgr2=0x7f4da006f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 shutdown_connections 2024-02-21T23:25:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.079+0000 7f4da4b8c640 1 -- 172.21.15.133:0/521165448 wait complete. 2024-02-21T23:25:57.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.084+0000 7faa3effd640 1 -- 172.21.15.133:0/4179646011 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa34035020 con 0x7faa400703c0 2024-02-21T23:25:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.084+0000 7faa3effd640 1 -- 172.21.15.133:0/4179646011 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa34031870 con 0x7faa400703c0 2024-02-21T23:25:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.084+0000 7faa3effd640 1 -- 172.21.15.133:0/4179646011 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa34031c40 con 0x7faa400703c0 2024-02-21T23:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.085+0000 7faa464a2640 1 -- 172.21.15.133:0/4179646011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa400703c0 msgr2=0x7faa400c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.085+0000 7faa464a2640 1 --2- 172.21.15.133:0/4179646011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa400703c0 0x7faa400c90b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa40072a30 tx=0x7faa3402b910 comp rx=0 tx=0).stop 2024-02-21T23:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.085+0000 7faa464a2640 1 -- 172.21.15.133:0/4179646011 shutdown_connections 2024-02-21T23:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.085+0000 7faa464a2640 1 --2- 172.21.15.133:0/4179646011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa400703c0 0x7faa400c90b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.085+0000 7faa464a2640 1 --2- 172.21.15.133:0/4179646011 >> v2:172.21.15.142:3300/0 conn(0x7faa40154240 0x7faa400ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.085+0000 7faa464a2640 1 -- 172.21.15.133:0/4179646011 >> 172.21.15.133:0/4179646011 conn(0x7faa40061b50 msgr2=0x7faa40072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.085+0000 7faa464a2640 1 -- 172.21.15.133:0/4179646011 shutdown_connections 2024-02-21T23:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.085+0000 7faa464a2640 1 -- 172.21.15.133:0/4179646011 wait complete. 2024-02-21T23:25:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.087+0000 7faa464a2640 1 Processor -- start 2024-02-21T23:25:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.087+0000 7faa464a2640 1 -- start start 2024-02-21T23:25:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.088+0000 7faa464a2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa400703c0 0x7faa4013f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.088+0000 7faa464a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa40070800 0x7faa40157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.088+0000 7faa464a2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa40154240 0x7faa401580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.088+0000 7faa464a2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa40154620 con 0x7faa40070800 2024-02-21T23:25:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.088+0000 7faa464a2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4015a9d0 con 0x7faa40154240 2024-02-21T23:25:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.088+0000 7faa464a2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4014fcd0 con 0x7faa400703c0 2024-02-21T23:25:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.089+0000 7faa3ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa400703c0 0x7faa4013f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.089+0000 7faa3ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa400703c0 0x7faa4013f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:46050/0 (socket says 172.21.15.133:46050) 2024-02-21T23:25:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.089+0000 7faa3ffff640 1 -- 172.21.15.133:0/3678972864 learned_addr learned my addr 172.21.15.133:0/3678972864 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.089+0000 7faa44a3f640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa40154240 0x7faa401580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.093+0000 7faa3ffff640 1 -- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa40154240 msgr2=0x7faa401580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.093+0000 7faa3ffff640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa40154240 0x7faa401580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.093+0000 7faa3ffff640 1 -- 172.21.15.133:0/3678972864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa40070800 msgr2=0x7faa40157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.093+0000 7faa3ffff640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa40070800 0x7faa40157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.093+0000 7faa3ffff640 1 -- 172.21.15.133:0/3678972864 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa40159aa0 con 0x7faa400703c0 2024-02-21T23:25:57.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.1 2024-02-21T23:25:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.096+0000 7faa3ffff640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa400703c0 0x7faa4013f170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa3000b4a0 tx=0x7faa3000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:57.082 DEBUG:teuthology.parallel:result is None 2024-02-21T23:25:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.099+0000 7faa3cff9640 1 -- 172.21.15.133:0/3678972864 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30014070 con 0x7faa400703c0 2024-02-21T23:25:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.099+0000 7faa3cff9640 1 -- 172.21.15.133:0/3678972864 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa30010610 con 0x7faa400703c0 2024-02-21T23:25:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.099+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4014f150 con 0x7faa400703c0 2024-02-21T23:25:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.099+0000 7faa3cff9640 1 -- 172.21.15.133:0/3678972864 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30010a00 con 0x7faa400703c0 2024-02-21T23:25:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.099+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4014f710 con 0x7faa400703c0 2024-02-21T23:25:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.100+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4006a130 con 0x7faa400703c0 2024-02-21T23:25:57.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.113+0000 7f15d1ffb640 1 -- 172.21.15.133:0/3246123184 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8035020 con 0x7f15d40c6f50 2024-02-21T23:25:57.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.113+0000 7f15d1ffb640 1 -- 172.21.15.133:0/3246123184 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8031830 con 0x7f15d40c6f50 2024-02-21T23:25:57.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.113+0000 7f15d1ffb640 1 -- 172.21.15.133:0/3246123184 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8031c00 con 0x7f15d40c6f50 2024-02-21T23:25:57.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.114+0000 7f15d971e640 1 -- 172.21.15.133:0/3246123184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d40c6f50 msgr2=0x7f15d40cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.114+0000 7f15d971e640 1 --2- 172.21.15.133:0/3246123184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d40c6f50 0x7f15d40cbc00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15c80097a0 tx=0x7f15c802b8d0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.114+0000 7f15d971e640 1 -- 172.21.15.133:0/3246123184 shutdown_connections 2024-02-21T23:25:57.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.114+0000 7f15d971e640 1 --2- 172.21.15.133:0/3246123184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d40c6f50 0x7f15d40cbc00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.114+0000 7f15d971e640 1 -- 172.21.15.133:0/3246123184 >> 172.21.15.133:0/3246123184 conn(0x7f15d4062000 msgr2=0x7f15d40cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.114+0000 7f15d971e640 1 -- 172.21.15.133:0/3246123184 shutdown_connections 2024-02-21T23:25:57.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.114+0000 7f15d971e640 1 -- 172.21.15.133:0/3246123184 wait complete. 2024-02-21T23:25:57.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.116+0000 7f15d971e640 1 Processor -- start 2024-02-21T23:25:57.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.116+0000 7f15d971e640 1 -- start start 2024-02-21T23:25:57.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.117+0000 7f15d971e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40c6f50 0x7f15d40c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.117+0000 7f15d971e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d40c7c10 0x7f15d4147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.117+0000 7f15d971e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d4154600 0x7f15d41481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.117+0000 7f15d971e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40b5720 con 0x7f15d40c7c10 2024-02-21T23:25:57.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.117+0000 7f15d971e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40c7780 con 0x7f15d4154600 2024-02-21T23:25:57.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.117+0000 7f15d971e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40b58a0 con 0x7f15d40c6f50 2024-02-21T23:25:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.118+0000 7f15d37fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d4154600 0x7f15d41481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.118+0000 7f15d37fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d4154600 0x7f15d41481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42840/0 (socket says 172.21.15.133:42840) 2024-02-21T23:25:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.118+0000 7f15d37fe640 1 -- 172.21.15.133:0/518406793 learned_addr learned my addr 172.21.15.133:0/518406793 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.121+0000 7f15d37fe640 1 -- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40c6f50 msgr2=0x7f15d40c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.121+0000 7f15d37fe640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40c6f50 0x7f15d40c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.121+0000 7f15d37fe640 1 -- 172.21.15.133:0/518406793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d40c7c10 msgr2=0x7f15d4147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.121+0000 7f15d37fe640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d40c7c10 0x7f15d4147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.121+0000 7f15d37fe640 1 -- 172.21.15.133:0/518406793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40c5e10 con 0x7f15d4154600 2024-02-21T23:25:57.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.124+0000 7f15d37fe640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d4154600 0x7f15d41481c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00cbf0 tx=0x7f15cc00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.127+0000 7f15bb7fe640 1 -- 172.21.15.133:0/518406793 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc017070 con 0x7f15d4154600 2024-02-21T23:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.127+0000 7f15bb7fe640 1 -- 172.21.15.133:0/518406793 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc00ace0 con 0x7f15d4154600 2024-02-21T23:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.127+0000 7f15bb7fe640 1 -- 172.21.15.133:0/518406793 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc012600 con 0x7f15d4154600 2024-02-21T23:25:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.127+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d40c60b0 con 0x7f15d4154600 2024-02-21T23:25:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.128+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d4157790 con 0x7f15d4154600 2024-02-21T23:25:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.129+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d406a130 con 0x7f15d4154600 2024-02-21T23:25:57.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.146+0000 7f15bb7fe640 1 -- 172.21.15.133:0/518406793 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15cc01a070 con 0x7f15d4154600 2024-02-21T23:25:57.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.147+0000 7f15bb7fe640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f15c006b520 0x7f15c006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.147+0000 7f15d2ffd640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f15c006b520 0x7f15c006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.148+0000 7f15d2ffd640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f15c006b520 0x7f15c006d9e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f15d40c2e70 tx=0x7f15c4007990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:57.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.218+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f39bc0c9d10 con 0x7f39bc1587b0 2024-02-21T23:25:57.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.234+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd97006a130 con 0x7fd970072fe0 2024-02-21T23:25:57.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.240+0000 7f39b2ffd640 1 -- 172.21.15.133:0/1609713176 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f39b4010040 con 0x7f39bc1587b0 2024-02-21T23:25:57.226 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f399c06b550 msgr2=0x7f399c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f399c06b550 0x7f399c06da10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39ac036000 tx=0x7f39ac030000 comp rx=0 tx=0).stop 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1587b0 msgr2=0x7f39bc1ef3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1587b0 0x7f39bc1ef3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39b400eba0 tx=0x7f39b400c490 comp rx=0 tx=0).stop 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 shutdown_connections 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f399c06b550 0x7f399c06da10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1587b0 0x7f39bc1ef3f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39bc0c63b0 0x7f39bc1f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 --2- 172.21.15.133:0/1609713176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc0c5fd0 0x7f39bc1f0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 >> 172.21.15.133:0/1609713176 conn(0x7f39bc061b50 msgr2=0x7f39bc1580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 shutdown_connections 2024-02-21T23:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.243+0000 7f39c0b1e640 1 -- 172.21.15.133:0/1609713176 wait complete. 2024-02-21T23:25:57.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.6 2024-02-21T23:25:57.240 DEBUG:teuthology.parallel:result is None 2024-02-21T23:25:57.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.257+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f829806db40 con 0x7f82980cfe10 2024-02-21T23:25:57.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.356+0000 7faa3cff9640 1 -- 172.21.15.133:0/3678972864 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa30017070 con 0x7faa400703c0 2024-02-21T23:25:57.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.357+0000 7faa3cff9640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7faa2c06b500 0x7faa2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.357+0000 7faa3f7fe640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7faa2c06b500 0x7faa2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.358+0000 7faa3f7fe640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7faa2c06b500 0x7faa2c06d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faa34036000 tx=0x7faa34030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:57.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.361+0000 7faa3cff9640 1 -- 172.21.15.133:0/3678972864 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7faa30056910 con 0x7faa400703c0 2024-02-21T23:25:57.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.379+0000 7fd9577fe640 1 -- 172.21.15.133:0/1153925734 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd96401f020 con 0x7fd970072fe0 2024-02-21T23:25:57.365 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:57.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.380+0000 7faa3cff9640 1 -- 172.21.15.133:0/3678972864 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7faa40159aa0 con 0x7faa400703c0 2024-02-21T23:25:57.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd95c06b4d0 msgr2=0x7fd95c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd95c06b4d0 0x7fd95c06d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd960006a90 tx=0x7fd960006a20 comp rx=0 tx=0).stop 2024-02-21T23:25:57.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072fe0 msgr2=0x7fd9701f0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072fe0 0x7fd9701f0840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd96400b4d0 tx=0x7fd96400b990 comp rx=0 tx=0).stop 2024-02-21T23:25:57.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 shutdown_connections 2024-02-21T23:25:57.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd95c06b4d0 0x7fd95c06d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9701542a0 0x7fd9701f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd970072fe0 0x7fd9701f0840 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 --2- 172.21.15.133:0/1153925734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd970072900 0x7fd9700c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 >> 172.21.15.133:0/1153925734 conn(0x7fd97006f6e0 msgr2=0x7fd9700c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 shutdown_connections 2024-02-21T23:25:57.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.381+0000 7fd9753fa640 1 -- 172.21.15.133:0/1153925734 wait complete. 2024-02-21T23:25:57.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.389+0000 7f15bb7fe640 1 -- 172.21.15.133:0/518406793 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f15cc056c90 con 0x7f15d4154600 2024-02-21T23:25:57.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.7 2024-02-21T23:25:57.378 DEBUG:teuthology.parallel:result is None 2024-02-21T23:25:57.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.406+0000 7f15bb7fe640 1 -- 172.21.15.133:0/518406793 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f15cc058bf0 con 0x7f15d4154600 2024-02-21T23:25:57.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.416+0000 7f828dffb640 1 -- 172.21.15.133:0/2845279153 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f828800f2e0 con 0x7f82980cfe10 2024-02-21T23:25:57.402 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:57.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f827c06b4d0 msgr2=0x7f827c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f827c06b4d0 0x7f827c06d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f829400ab90 tx=0x7f8294009250 comp rx=0 tx=0).stop 2024-02-21T23:25:57.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cfe10 msgr2=0x7f82981f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cfe10 0x7f82981f3130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f828800b3e0 tx=0x7f828800b8a0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 shutdown_connections 2024-02-21T23:25:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f827c06b4d0 0x7f827c06d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f829815a170 0x7f82980c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82980cfe10 0x7f82981f3130 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 --2- 172.21.15.133:0/2845279153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82980cc890 0x7f829814c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 >> 172.21.15.133:0/2845279153 conn(0x7f8298061b50 msgr2=0x7f829814e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 shutdown_connections 2024-02-21T23:25:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.419+0000 7f829f436640 1 -- 172.21.15.133:0/2845279153 wait complete. 2024-02-21T23:25:57.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.2 2024-02-21T23:25:57.416 DEBUG:teuthology.parallel:result is None 2024-02-21T23:25:57.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.573+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faa40157960 con 0x7faa400703c0 2024-02-21T23:25:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.577+0000 7faa3cff9640 1 -- 172.21.15.133:0/3678972864 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faa30058870 con 0x7faa400703c0 2024-02-21T23:25:57.563 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7faa2c06b500 msgr2=0x7faa2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7faa2c06b500 0x7faa2c06d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faa34036000 tx=0x7faa34030000 comp rx=0 tx=0).stop 2024-02-21T23:25:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa400703c0 msgr2=0x7faa4013f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa400703c0 0x7faa4013f170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa3000b4a0 tx=0x7faa3000b960 comp rx=0 tx=0).stop 2024-02-21T23:25:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 shutdown_connections 2024-02-21T23:25:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7faa2c06b500 0x7faa2c06d9c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa40154240 0x7faa401580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa40070800 0x7faa40157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 --2- 172.21.15.133:0/3678972864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa400703c0 0x7faa4013f170 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 >> 172.21.15.133:0/3678972864 conn(0x7faa40061b50 msgr2=0x7faa40150b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 shutdown_connections 2024-02-21T23:25:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.580+0000 7faa464a2640 1 -- 172.21.15.133:0/3678972864 wait complete. 2024-02-21T23:25:57.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.4 2024-02-21T23:25:57.578 DEBUG:teuthology.parallel:result is None 2024-02-21T23:25:57.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.605+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f15d41e5bb0 con 0x7f15d4154600 2024-02-21T23:25:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.609+0000 7f15bb7fe640 1 -- 172.21.15.133:0/518406793 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15cc05cc90 con 0x7f15d4154600 2024-02-21T23:25:57.595 INFO:teuthology.orchestra.run.smithi133.stdout:55834574862 2024-02-21T23:25:57.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.611+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f15c006b520 msgr2=0x7f15c006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.611+0000 7f15d971e640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f15c006b520 0x7f15c006d9e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f15d40c2e70 tx=0x7f15c4007990 comp rx=0 tx=0).stop 2024-02-21T23:25:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.611+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d4154600 msgr2=0x7f15d41481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.611+0000 7f15d971e640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d4154600 0x7f15d41481c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00cbf0 tx=0x7f15cc00a430 comp rx=0 tx=0).stop 2024-02-21T23:25:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.612+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 shutdown_connections 2024-02-21T23:25:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.612+0000 7f15d971e640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f15c006b520 0x7f15c006d9e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.612+0000 7f15d971e640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d4154600 0x7f15d41481c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.612+0000 7f15d971e640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d40c7c10 0x7f15d4147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.612+0000 7f15d971e640 1 --2- 172.21.15.133:0/518406793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40c6f50 0x7f15d40c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.612+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 >> 172.21.15.133:0/518406793 conn(0x7f15d4062000 msgr2=0x7f15d40ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.612+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 shutdown_connections 2024-02-21T23:25:57.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.612+0000 7f15d971e640 1 -- 172.21.15.133:0/518406793 wait complete. 2024-02-21T23:25:57.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574862 got 55834574862 for osd.3 2024-02-21T23:25:57.609 DEBUG:teuthology.parallel:result is None 2024-02-21T23:25:57.609 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-02-21T23:25:57.609 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-21T23:25:57.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.764+0000 7f241aaa2640 1 Processor -- start 2024-02-21T23:25:57.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.764+0000 7f241aaa2640 1 -- start start 2024-02-21T23:25:57.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.765+0000 7f241aaa2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24140ce3c0 0x7f24140c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.765+0000 7f241aaa2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f241414e510 0x7f241414e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.765+0000 7f241aaa2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241414eec0 con 0x7f24140cdfe0 2024-02-21T23:25:57.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.765+0000 7f241aaa2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24140c84b0 con 0x7f241414e510 2024-02-21T23:25:57.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.765+0000 7f241aaa2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2414158ca0 con 0x7f24140ce3c0 2024-02-21T23:25:57.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.766+0000 7f241903f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f241414e510 0x7f241414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.766+0000 7f241883e640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f24140cdfe0 0x7f2414070d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:37460/0 (socket says 172.21.15.133:37460) 2024-02-21T23:25:57.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.766+0000 7f241883e640 1 -- 172.21.15.133:0/2825088172 learned_addr learned my addr 172.21.15.133:0/2825088172 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:57.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.766+0000 7f2413fff640 1 --2- 172.21.15.133:0/2825088172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24140ce3c0 0x7f24140c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.768+0000 7f241903f640 1 -- 172.21.15.133:0/2825088172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24140ce3c0 msgr2=0x7f24140c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:57.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.768+0000 7f241903f640 1 --2- 172.21.15.133:0/2825088172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24140ce3c0 0x7f24140c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.768+0000 7f241903f640 1 -- 172.21.15.133:0/2825088172 >> v1:172.21.15.142:6789/0 conn(0x7f24140cdfe0 legacy=0x7f2414070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.768+0000 7f241903f640 1 -- 172.21.15.133:0/2825088172 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140717e0 con 0x7f241414e510 2024-02-21T23:25:57.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.770+0000 7f241903f640 1 --2- 172.21.15.133:0/2825088172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241414e510 0x7f241414e980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f240c009d90 tx=0x7f240c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8bcfe5ffd0914f server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f24137fe640 1 -- 172.21.15.133:0/2825088172 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c039070 con 0x7f241414e510 2024-02-21T23:25:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f24137fe640 1 -- 172.21.15.133:0/2825088172 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c02cc50 con 0x7f241414e510 2024-02-21T23:25:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f24137fe640 1 -- 172.21.15.133:0/2825088172 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c034440 con 0x7f241414e510 2024-02-21T23:25:57.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f241aaa2640 1 -- 172.21.15.133:0/2825088172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241414e510 msgr2=0x7f241414e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f241aaa2640 1 --2- 172.21.15.133:0/2825088172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241414e510 0x7f241414e980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f240c009d90 tx=0x7f240c02bea0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f241aaa2640 1 -- 172.21.15.133:0/2825088172 shutdown_connections 2024-02-21T23:25:57.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f241aaa2640 1 --2- 172.21.15.133:0/2825088172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241414e510 0x7f241414e980 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f241aaa2640 1 --2- 172.21.15.133:0/2825088172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24140ce3c0 0x7f24140c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f241aaa2640 1 -- 172.21.15.133:0/2825088172 >> 172.21.15.133:0/2825088172 conn(0x7f241406aec0 msgr2=0x7f241406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:57.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.774+0000 7f241aaa2640 1 -- 172.21.15.133:0/2825088172 shutdown_connections 2024-02-21T23:25:57.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.775+0000 7f241aaa2640 1 -- 172.21.15.133:0/2825088172 wait complete. 2024-02-21T23:25:57.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.776+0000 7f241aaa2640 1 Processor -- start 2024-02-21T23:25:57.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.776+0000 7f241aaa2640 1 -- start start 2024-02-21T23:25:57.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.776+0000 7f241aaa2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24140cdfe0 0x7f241414c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.776+0000 7f241aaa2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24140ce3c0 0x7f2414154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.776+0000 7f241aaa2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f241414e510 0x7f24141551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.776+0000 7f241aaa2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2414158650 con 0x7f24140ce3c0 2024-02-21T23:25:57.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.777+0000 7f241aaa2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2414158370 con 0x7f24140cdfe0 2024-02-21T23:25:57.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.777+0000 7f241aaa2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2414158930 con 0x7f241414e510 2024-02-21T23:25:57.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.777+0000 7f2413fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24140ce3c0 0x7f2414154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.777+0000 7f241883e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24140cdfe0 0x7f241414c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.777+0000 7f241903f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f241414e510 0x7f24141551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.777+0000 7f2413fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24140ce3c0 0x7f2414154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40254/0 (socket says 172.21.15.133:40254) 2024-02-21T23:25:57.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.777+0000 7f2413fff640 1 -- 172.21.15.133:0/3983502578 learned_addr learned my addr 172.21.15.133:0/3983502578 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:57.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.781+0000 7f2413fff640 1 -- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f241414e510 msgr2=0x7f24141551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.781+0000 7f2413fff640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f241414e510 0x7f24141551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.781+0000 7f2413fff640 1 -- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24140cdfe0 msgr2=0x7f241414c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:57.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.781+0000 7f2413fff640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24140cdfe0 0x7f241414c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:57.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.781+0000 7f2413fff640 1 -- 172.21.15.133:0/3983502578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140c57a0 con 0x7f24140ce3c0 2024-02-21T23:25:57.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.783+0000 7f2413fff640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24140ce3c0 0x7f2414154c90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f240800e9b0 tx=0x7f240800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.786+0000 7f24117fa640 1 -- 172.21.15.133:0/3983502578 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408017070 con 0x7f24140ce3c0 2024-02-21T23:25:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.786+0000 7f24117fa640 1 -- 172.21.15.133:0/3983502578 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240800c9a0 con 0x7f24140ce3c0 2024-02-21T23:25:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.786+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24141ed650 con 0x7f24140ce3c0 2024-02-21T23:25:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.786+0000 7f24117fa640 1 -- 172.21.15.133:0/3983502578 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240800cd90 con 0x7f24140ce3c0 2024-02-21T23:25:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.787+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24141edc10 con 0x7f24140ce3c0 2024-02-21T23:25:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.787+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241414ce00 con 0x7f24140ce3c0 2024-02-21T23:25:57.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.806+0000 7f24117fa640 1 -- 172.21.15.133:0/3983502578 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f240801a070 con 0x7f24140ce3c0 2024-02-21T23:25:57.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.806+0000 7f24117fa640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f240006b550 0x7f240006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:57.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.806+0000 7f241883e640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f240006b550 0x7f240006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:57.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:57.807+0000 7f241883e640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f240006b550 0x7f240006da10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2404000f60 tx=0x7f2404007e80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.043+0000 7f24117fa640 1 -- 172.21.15.133:0/3983502578 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f24080571c0 con 0x7f24140ce3c0 2024-02-21T23:25:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.043+0000 7f24117fa640 1 -- 172.21.15.133:0/3983502578 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f241414ce00 con 0x7f24140ce3c0 2024-02-21T23:25:58.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.222+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24141ee2d0 con 0x7f240006b550 2024-02-21T23:25:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.230+0000 7f24117fa640 1 -- 172.21.15.133:0/3983502578 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332611 (secure 0 0 0) 0x7f24141ee2d0 con 0x7f240006b550 2024-02-21T23:25:58.222 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:58.225 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-02-21T23:25:58.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f240006b550 msgr2=0x7f240006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:58.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f240006b550 0x7f240006da10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2404000f60 tx=0x7f2404007e80 comp rx=0 tx=0).stop 2024-02-21T23:25:58.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24140ce3c0 msgr2=0x7f2414154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:58.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24140ce3c0 0x7f2414154c90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f240800e9b0 tx=0x7f240800ee70 comp rx=0 tx=0).stop 2024-02-21T23:25:58.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 shutdown_connections 2024-02-21T23:25:58.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f240006b550 0x7f240006da10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f241414e510 0x7f24141551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24140ce3c0 0x7f2414154c90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 --2- 172.21.15.133:0/3983502578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24140cdfe0 0x7f241414c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 >> 172.21.15.133:0/3983502578 conn(0x7f241406aec0 msgr2=0x7f241415bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:58.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 shutdown_connections 2024-02-21T23:25:58.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.234+0000 7f241aaa2640 1 -- 172.21.15.133:0/3983502578 wait complete. 2024-02-21T23:25:58.231 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2024-02-21T23:25:57.526390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218332,"kb_used_data":2176,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756388,"statfs":{"total":773094113280,"available":772870541312,"internally_reserved":0,"allocated":2228224,"data_stored":1992064,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.835369"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637827+0000","last_change":"2024-02-21T23:25:16.134390+0000","last_active":"2024-02-21T23:25:22.637827+0000","last_peered":"2024-02-21T23:25:22.637827+0000","last_clean":"2024-02-21T23:25:22.637827+0000","last_became_active":"2024-02-21T23:25:16.134112+0000","last_became_peered":"2024-02-21T23:25:16.134112+0000","last_unstale":"2024-02-21T23:25:22.637827+0000","last_undegraded":"2024-02-21T23:25:22.637827+0000","last_fullsized":"2024-02-21T23:25:22.637827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195309+0000","last_change":"2024-02-21T23:25:17.135738+0000","last_active":"2024-02-21T23:25:23.195309+0000","last_peered":"2024-02-21T23:25:23.195309+0000","last_clean":"2024-02-21T23:25:23.195309+0000","last_became_active":"2024-02-21T23:25:17.135586+0000","last_became_peered":"2024-02-21T23:25:17.135586+0000","last_unstale":"2024-02-21T23:25:23.195309+0000","last_undegraded":"2024-02-21T23:25:23.195309+0000","last_fullsized":"2024-02-21T23:25:23.195309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194873+0000","last_change":"2024-02-21T23:25:22.248962+0000","last_active":"2024-02-21T23:25:23.194873+0000","last_peered":"2024-02-21T23:25:23.194873+0000","last_clean":"2024-02-21T23:25:23.194873+0000","last_became_active":"2024-02-21T23:25:22.248610+0000","last_became_peered":"2024-02-21T23:25:22.248610+0000","last_unstale":"2024-02-21T23:25:23.194873+0000","last_undegraded":"2024-02-21T23:25:23.194873+0000","last_fullsized":"2024-02-21T23:25:23.194873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:45.939036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195369+0000","last_change":"2024-02-21T23:25:16.123635+0000","last_active":"2024-02-21T23:25:23.195369+0000","last_peered":"2024-02-21T23:25:23.195369+0000","last_clean":"2024-02-21T23:25:23.195369+0000","last_became_active":"2024-02-21T23:25:16.123374+0000","last_became_peered":"2024-02-21T23:25:16.123374+0000","last_unstale":"2024-02-21T23:25:23.195369+0000","last_undegraded":"2024-02-21T23:25:23.195369+0000","last_fullsized":"2024-02-21T23:25:23.195369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637647+0000","last_change":"2024-02-21T23:25:17.192858+0000","last_active":"2024-02-21T23:25:22.637647+0000","last_peered":"2024-02-21T23:25:22.637647+0000","last_clean":"2024-02-21T23:25:22.637647+0000","last_became_active":"2024-02-21T23:25:17.192626+0000","last_became_peered":"2024-02-21T23:25:17.192626+0000","last_unstale":"2024-02-21T23:25:22.637647+0000","last_undegraded":"2024-02-21T23:25:22.637647+0000","last_fullsized":"2024-02-21T23:25:22.637647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195812+0000","last_change":"2024-02-21T23:25:21.683490+0000","last_active":"2024-02-21T23:25:23.195812+0000","last_peered":"2024-02-21T23:25:23.195812+0000","last_clean":"2024-02-21T23:25:23.195812+0000","last_became_active":"2024-02-21T23:25:21.683173+0000","last_became_peered":"2024-02-21T23:25:21.683173+0000","last_unstale":"2024-02-21T23:25:23.195812+0000","last_undegraded":"2024-02-21T23:25:23.195812+0000","last_fullsized":"2024-02-21T23:25:23.195812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:49:01.803167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196175+0000","last_change":"2024-02-21T23:25:16.134805+0000","last_active":"2024-02-21T23:25:23.196175+0000","last_peered":"2024-02-21T23:25:23.196175+0000","last_clean":"2024-02-21T23:25:23.196175+0000","last_became_active":"2024-02-21T23:25:16.134533+0000","last_became_peered":"2024-02-21T23:25:16.134533+0000","last_unstale":"2024-02-21T23:25:23.196175+0000","last_undegraded":"2024-02-21T23:25:23.196175+0000","last_fullsized":"2024-02-21T23:25:23.196175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197229+0000","last_change":"2024-02-21T23:25:17.182131+0000","last_active":"2024-02-21T23:25:23.197229+0000","last_peered":"2024-02-21T23:25:23.197229+0000","last_clean":"2024-02-21T23:25:23.197229+0000","last_became_active":"2024-02-21T23:25:17.181886+0000","last_became_peered":"2024-02-21T23:25:17.181886+0000","last_unstale":"2024-02-21T23:25:23.197229+0000","last_undegraded":"2024-02-21T23:25:23.197229+0000","last_fullsized":"2024-02-21T23:25:23.197229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195843+0000","last_change":"2024-02-21T23:25:21.683506+0000","last_active":"2024-02-21T23:25:23.195843+0000","last_peered":"2024-02-21T23:25:23.195843+0000","last_clean":"2024-02-21T23:25:23.195843+0000","last_became_active":"2024-02-21T23:25:21.683115+0000","last_became_peered":"2024-02-21T23:25:21.683115+0000","last_unstale":"2024-02-21T23:25:23.195843+0000","last_undegraded":"2024-02-21T23:25:23.195843+0000","last_fullsized":"2024-02-21T23:25:23.195843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:13.865145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195380+0000","last_change":"2024-02-21T23:25:16.130851+0000","last_active":"2024-02-21T23:25:23.195380+0000","last_peered":"2024-02-21T23:25:23.195380+0000","last_clean":"2024-02-21T23:25:23.195380+0000","last_became_active":"2024-02-21T23:25:16.130604+0000","last_became_peered":"2024-02-21T23:25:16.130604+0000","last_unstale":"2024-02-21T23:25:23.195380+0000","last_undegraded":"2024-02-21T23:25:23.195380+0000","last_fullsized":"2024-02-21T23:25:23.195380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197689+0000","last_change":"2024-02-21T23:25:17.192642+0000","last_active":"2024-02-21T23:25:23.197689+0000","last_peered":"2024-02-21T23:25:23.197689+0000","last_clean":"2024-02-21T23:25:23.197689+0000","last_became_active":"2024-02-21T23:25:17.192394+0000","last_became_peered":"2024-02-21T23:25:17.192394+0000","last_unstale":"2024-02-21T23:25:23.197689+0000","last_undegraded":"2024-02-21T23:25:23.197689+0000","last_fullsized":"2024-02-21T23:25:23.197689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195564+0000","last_change":"2024-02-21T23:25:21.679491+0000","last_active":"2024-02-21T23:25:23.195564+0000","last_peered":"2024-02-21T23:25:23.195564+0000","last_clean":"2024-02-21T23:25:23.195564+0000","last_became_active":"2024-02-21T23:25:21.679117+0000","last_became_peered":"2024-02-21T23:25:21.679117+0000","last_unstale":"2024-02-21T23:25:23.195564+0000","last_undegraded":"2024-02-21T23:25:23.195564+0000","last_fullsized":"2024-02-21T23:25:23.195564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:52.680971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670455+0000","last_change":"2024-02-21T23:25:17.161438+0000","last_active":"2024-02-21T23:25:22.670455+0000","last_peered":"2024-02-21T23:25:22.670455+0000","last_clean":"2024-02-21T23:25:22.670455+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:25:22.670455+0000","last_undegraded":"2024-02-21T23:25:22.670455+0000","last_fullsized":"2024-02-21T23:25:22.670455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196550+0000","last_change":"2024-02-21T23:25:16.136913+0000","last_active":"2024-02-21T23:25:23.196550+0000","last_peered":"2024-02-21T23:25:23.196550+0000","last_clean":"2024-02-21T23:25:23.196550+0000","last_became_active":"2024-02-21T23:25:16.136628+0000","last_became_peered":"2024-02-21T23:25:16.136628+0000","last_unstale":"2024-02-21T23:25:23.196550+0000","last_undegraded":"2024-02-21T23:25:23.196550+0000","last_fullsized":"2024-02-21T23:25:23.196550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196588+0000","last_change":"2024-02-21T23:25:21.683706+0000","last_active":"2024-02-21T23:25:23.196588+0000","last_peered":"2024-02-21T23:25:23.196588+0000","last_clean":"2024-02-21T23:25:23.196588+0000","last_became_active":"2024-02-21T23:25:21.683116+0000","last_became_peered":"2024-02-21T23:25:21.683116+0000","last_unstale":"2024-02-21T23:25:23.196588+0000","last_undegraded":"2024-02-21T23:25:23.196588+0000","last_fullsized":"2024-02-21T23:25:23.196588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:06.265383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195016+0000","last_change":"2024-02-21T23:25:17.137829+0000","last_active":"2024-02-21T23:25:23.195016+0000","last_peered":"2024-02-21T23:25:23.195016+0000","last_clean":"2024-02-21T23:25:23.195016+0000","last_became_active":"2024-02-21T23:25:17.137557+0000","last_became_peered":"2024-02-21T23:25:17.137557+0000","last_unstale":"2024-02-21T23:25:23.195016+0000","last_undegraded":"2024-02-21T23:25:23.195016+0000","last_fullsized":"2024-02-21T23:25:23.195016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723448+0000","last_change":"2024-02-21T23:25:16.146181+0000","last_active":"2024-02-21T23:25:22.723448+0000","last_peered":"2024-02-21T23:25:22.723448+0000","last_clean":"2024-02-21T23:25:22.723448+0000","last_became_active":"2024-02-21T23:25:16.145932+0000","last_became_peered":"2024-02-21T23:25:16.145932+0000","last_unstale":"2024-02-21T23:25:22.723448+0000","last_undegraded":"2024-02-21T23:25:22.723448+0000","last_fullsized":"2024-02-21T23:25:22.723448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196544+0000","last_change":"2024-02-21T23:25:21.686044+0000","last_active":"2024-02-21T23:25:23.196544+0000","last_peered":"2024-02-21T23:25:23.196544+0000","last_clean":"2024-02-21T23:25:23.196544+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:25:23.196544+0000","last_undegraded":"2024-02-21T23:25:23.196544+0000","last_fullsized":"2024-02-21T23:25:23.196544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:39.740454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195078+0000","last_change":"2024-02-21T23:25:17.190236+0000","last_active":"2024-02-21T23:25:23.195078+0000","last_peered":"2024-02-21T23:25:23.195078+0000","last_clean":"2024-02-21T23:25:23.195078+0000","last_became_active":"2024-02-21T23:25:17.189957+0000","last_became_peered":"2024-02-21T23:25:17.189957+0000","last_unstale":"2024-02-21T23:25:23.195078+0000","last_undegraded":"2024-02-21T23:25:23.195078+0000","last_fullsized":"2024-02-21T23:25:23.195078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723142+0000","last_change":"2024-02-21T23:25:16.148108+0000","last_active":"2024-02-21T23:25:22.723142+0000","last_peered":"2024-02-21T23:25:22.723142+0000","last_clean":"2024-02-21T23:25:22.723142+0000","last_became_active":"2024-02-21T23:25:16.147930+0000","last_became_peered":"2024-02-21T23:25:16.147930+0000","last_unstale":"2024-02-21T23:25:22.723142+0000","last_undegraded":"2024-02-21T23:25:22.723142+0000","last_fullsized":"2024-02-21T23:25:22.723142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723970+0000","last_change":"2024-02-21T23:25:22.248150+0000","last_active":"2024-02-21T23:25:22.723970+0000","last_peered":"2024-02-21T23:25:22.723970+0000","last_clean":"2024-02-21T23:25:22.723970+0000","last_became_active":"2024-02-21T23:25:22.247769+0000","last_became_peered":"2024-02-21T23:25:22.247769+0000","last_unstale":"2024-02-21T23:25:22.723970+0000","last_undegraded":"2024-02-21T23:25:22.723970+0000","last_fullsized":"2024-02-21T23:25:22.723970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196252+0000","last_change":"2024-02-21T23:25:17.189758+0000","last_active":"2024-02-21T23:25:23.196252+0000","last_peered":"2024-02-21T23:25:23.196252+0000","last_clean":"2024-02-21T23:25:23.196252+0000","last_became_active":"2024-02-21T23:25:17.189492+0000","last_became_peered":"2024-02-21T23:25:17.189492+0000","last_unstale":"2024-02-21T23:25:23.196252+0000","last_undegraded":"2024-02-21T23:25:23.196252+0000","last_fullsized":"2024-02-21T23:25:23.196252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:42.526598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195319+0000","last_change":"2024-02-21T23:25:16.131611+0000","last_active":"2024-02-21T23:25:23.195319+0000","last_peered":"2024-02-21T23:25:23.195319+0000","last_clean":"2024-02-21T23:25:23.195319+0000","last_became_active":"2024-02-21T23:25:16.131158+0000","last_became_peered":"2024-02-21T23:25:16.131158+0000","last_unstale":"2024-02-21T23:25:23.195319+0000","last_undegraded":"2024-02-21T23:25:23.195319+0000","last_fullsized":"2024-02-21T23:25:23.195319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196480+0000","last_change":"2024-02-21T23:25:21.683821+0000","last_active":"2024-02-21T23:25:23.196480+0000","last_peered":"2024-02-21T23:25:23.196480+0000","last_clean":"2024-02-21T23:25:23.196480+0000","last_became_active":"2024-02-21T23:25:21.683466+0000","last_became_peered":"2024-02-21T23:25:21.683466+0000","last_unstale":"2024-02-21T23:25:23.196480+0000","last_undegraded":"2024-02-21T23:25:23.196480+0000","last_fullsized":"2024-02-21T23:25:23.196480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637653+0000","last_change":"2024-02-21T23:25:17.191394+0000","last_active":"2024-02-21T23:25:22.637653+0000","last_peered":"2024-02-21T23:25:22.637653+0000","last_clean":"2024-02-21T23:25:22.637653+0000","last_became_active":"2024-02-21T23:25:17.191106+0000","last_became_peered":"2024-02-21T23:25:17.191106+0000","last_unstale":"2024-02-21T23:25:22.637653+0000","last_undegraded":"2024-02-21T23:25:22.637653+0000","last_fullsized":"2024-02-21T23:25:22.637653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670984+0000","last_change":"2024-02-21T23:25:16.134862+0000","last_active":"2024-02-21T23:25:22.670984+0000","last_peered":"2024-02-21T23:25:22.670984+0000","last_clean":"2024-02-21T23:25:22.670984+0000","last_became_active":"2024-02-21T23:25:16.134602+0000","last_became_peered":"2024-02-21T23:25:16.134602+0000","last_unstale":"2024-02-21T23:25:22.670984+0000","last_undegraded":"2024-02-21T23:25:22.670984+0000","last_fullsized":"2024-02-21T23:25:22.670984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196150+0000","last_change":"2024-02-21T23:25:22.249606+0000","last_active":"2024-02-21T23:25:23.196150+0000","last_peered":"2024-02-21T23:25:23.196150+0000","last_clean":"2024-02-21T23:25:23.196150+0000","last_became_active":"2024-02-21T23:25:22.249214+0000","last_became_peered":"2024-02-21T23:25:22.249214+0000","last_unstale":"2024-02-21T23:25:23.196150+0000","last_undegraded":"2024-02-21T23:25:23.196150+0000","last_fullsized":"2024-02-21T23:25:23.196150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:36.929076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195603+0000","last_change":"2024-02-21T23:25:17.183290+0000","last_active":"2024-02-21T23:25:23.195603+0000","last_peered":"2024-02-21T23:25:23.195603+0000","last_clean":"2024-02-21T23:25:23.195603+0000","last_became_active":"2024-02-21T23:25:17.183047+0000","last_became_peered":"2024-02-21T23:25:17.183047+0000","last_unstale":"2024-02-21T23:25:23.195603+0000","last_undegraded":"2024-02-21T23:25:23.195603+0000","last_fullsized":"2024-02-21T23:25:23.195603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723977+0000","last_change":"2024-02-21T23:25:16.146202+0000","last_active":"2024-02-21T23:25:22.723977+0000","last_peered":"2024-02-21T23:25:22.723977+0000","last_clean":"2024-02-21T23:25:22.723977+0000","last_became_active":"2024-02-21T23:25:16.145911+0000","last_became_peered":"2024-02-21T23:25:16.145911+0000","last_unstale":"2024-02-21T23:25:22.723977+0000","last_undegraded":"2024-02-21T23:25:22.723977+0000","last_fullsized":"2024-02-21T23:25:22.723977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194768+0000","last_change":"2024-02-21T23:25:22.243297+0000","last_active":"2024-02-21T23:25:23.194768+0000","last_peered":"2024-02-21T23:25:23.194768+0000","last_clean":"2024-02-21T23:25:23.194768+0000","last_became_active":"2024-02-21T23:25:22.242922+0000","last_became_peered":"2024-02-21T23:25:22.242922+0000","last_unstale":"2024-02-21T23:25:23.194768+0000","last_undegraded":"2024-02-21T23:25:23.194768+0000","last_fullsized":"2024-02-21T23:25:23.194768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:52:51.773574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723650+0000","last_change":"2024-02-21T23:25:17.190577+0000","last_active":"2024-02-21T23:25:22.723650+0000","last_peered":"2024-02-21T23:25:22.723650+0000","last_clean":"2024-02-21T23:25:22.723650+0000","last_became_active":"2024-02-21T23:25:17.190332+0000","last_became_peered":"2024-02-21T23:25:17.190332+0000","last_unstale":"2024-02-21T23:25:22.723650+0000","last_undegraded":"2024-02-21T23:25:22.723650+0000","last_fullsized":"2024-02-21T23:25:22.723650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196263+0000","last_change":"2024-02-21T23:25:16.134076+0000","last_active":"2024-02-21T23:25:23.196263+0000","last_peered":"2024-02-21T23:25:23.196263+0000","last_clean":"2024-02-21T23:25:23.196263+0000","last_became_active":"2024-02-21T23:25:16.133816+0000","last_became_peered":"2024-02-21T23:25:16.133816+0000","last_unstale":"2024-02-21T23:25:23.196263+0000","last_undegraded":"2024-02-21T23:25:23.196263+0000","last_fullsized":"2024-02-21T23:25:23.196263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637332+0000","last_change":"2024-02-21T23:25:22.243360+0000","last_active":"2024-02-21T23:25:22.637332+0000","last_peered":"2024-02-21T23:25:22.637332+0000","last_clean":"2024-02-21T23:25:22.637332+0000","last_became_active":"2024-02-21T23:25:22.242958+0000","last_became_peered":"2024-02-21T23:25:22.242958+0000","last_unstale":"2024-02-21T23:25:22.637332+0000","last_undegraded":"2024-02-21T23:25:22.637332+0000","last_fullsized":"2024-02-21T23:25:22.637332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197769+0000","last_change":"2024-02-21T23:25:17.192402+0000","last_active":"2024-02-21T23:25:23.197769+0000","last_peered":"2024-02-21T23:25:23.197769+0000","last_clean":"2024-02-21T23:25:23.197769+0000","last_became_active":"2024-02-21T23:25:17.192138+0000","last_became_peered":"2024-02-21T23:25:17.192138+0000","last_unstale":"2024-02-21T23:25:23.197769+0000","last_undegraded":"2024-02-21T23:25:23.197769+0000","last_fullsized":"2024-02-21T23:25:23.197769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723671+0000","last_change":"2024-02-21T23:25:16.145521+0000","last_active":"2024-02-21T23:25:22.723671+0000","last_peered":"2024-02-21T23:25:22.723671+0000","last_clean":"2024-02-21T23:25:22.723671+0000","last_became_active":"2024-02-21T23:25:16.145255+0000","last_became_peered":"2024-02-21T23:25:16.145255+0000","last_unstale":"2024-02-21T23:25:22.723671+0000","last_undegraded":"2024-02-21T23:25:22.723671+0000","last_fullsized":"2024-02-21T23:25:22.723671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723238+0000","last_change":"2024-02-21T23:25:21.683849+0000","last_active":"2024-02-21T23:25:22.723238+0000","last_peered":"2024-02-21T23:25:22.723238+0000","last_clean":"2024-02-21T23:25:22.723238+0000","last_became_active":"2024-02-21T23:25:21.683379+0000","last_became_peered":"2024-02-21T23:25:21.683379+0000","last_unstale":"2024-02-21T23:25:22.723238+0000","last_undegraded":"2024-02-21T23:25:22.723238+0000","last_fullsized":"2024-02-21T23:25:22.723238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196877+0000","last_change":"2024-02-21T23:25:17.166232+0000","last_active":"2024-02-21T23:25:23.196877+0000","last_peered":"2024-02-21T23:25:23.196877+0000","last_clean":"2024-02-21T23:25:23.196877+0000","last_became_active":"2024-02-21T23:25:17.165999+0000","last_became_peered":"2024-02-21T23:25:17.165999+0000","last_unstale":"2024-02-21T23:25:23.196877+0000","last_undegraded":"2024-02-21T23:25:23.196877+0000","last_fullsized":"2024-02-21T23:25:23.196877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:45.125986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197593+0000","last_change":"2024-02-21T23:25:16.147082+0000","last_active":"2024-02-21T23:25:23.197593+0000","last_peered":"2024-02-21T23:25:23.197593+0000","last_clean":"2024-02-21T23:25:23.197593+0000","last_became_active":"2024-02-21T23:25:16.146797+0000","last_became_peered":"2024-02-21T23:25:16.146797+0000","last_unstale":"2024-02-21T23:25:23.197593+0000","last_undegraded":"2024-02-21T23:25:23.197593+0000","last_fullsized":"2024-02-21T23:25:23.197593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723612+0000","last_change":"2024-02-21T23:25:21.682860+0000","last_active":"2024-02-21T23:25:22.723612+0000","last_peered":"2024-02-21T23:25:22.723612+0000","last_clean":"2024-02-21T23:25:22.723612+0000","last_became_active":"2024-02-21T23:25:21.682404+0000","last_became_peered":"2024-02-21T23:25:21.682404+0000","last_unstale":"2024-02-21T23:25:22.723612+0000","last_undegraded":"2024-02-21T23:25:22.723612+0000","last_fullsized":"2024-02-21T23:25:22.723612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723262+0000","last_change":"2024-02-21T23:25:17.190425+0000","last_active":"2024-02-21T23:25:22.723262+0000","last_peered":"2024-02-21T23:25:22.723262+0000","last_clean":"2024-02-21T23:25:22.723262+0000","last_became_active":"2024-02-21T23:25:17.190172+0000","last_became_peered":"2024-02-21T23:25:17.190172+0000","last_unstale":"2024-02-21T23:25:22.723262+0000","last_undegraded":"2024-02-21T23:25:22.723262+0000","last_fullsized":"2024-02-21T23:25:22.723262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636897+0000","last_change":"2024-02-21T23:25:16.108263+0000","last_active":"2024-02-21T23:25:22.636897+0000","last_peered":"2024-02-21T23:25:22.636897+0000","last_clean":"2024-02-21T23:25:22.636897+0000","last_became_active":"2024-02-21T23:25:16.107822+0000","last_became_peered":"2024-02-21T23:25:16.107822+0000","last_unstale":"2024-02-21T23:25:22.636897+0000","last_undegraded":"2024-02-21T23:25:22.636897+0000","last_fullsized":"2024-02-21T23:25:22.636897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723957+0000","last_change":"2024-02-21T23:25:21.686034+0000","last_active":"2024-02-21T23:25:22.723957+0000","last_peered":"2024-02-21T23:25:22.723957+0000","last_clean":"2024-02-21T23:25:22.723957+0000","last_became_active":"2024-02-21T23:25:21.685546+0000","last_became_peered":"2024-02-21T23:25:21.685546+0000","last_unstale":"2024-02-21T23:25:22.723957+0000","last_undegraded":"2024-02-21T23:25:22.723957+0000","last_fullsized":"2024-02-21T23:25:22.723957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636614+0000","last_change":"2024-02-21T23:25:17.161738+0000","last_active":"2024-02-21T23:25:22.636614+0000","last_peered":"2024-02-21T23:25:22.636614+0000","last_clean":"2024-02-21T23:25:22.636614+0000","last_became_active":"2024-02-21T23:25:17.161459+0000","last_became_peered":"2024-02-21T23:25:17.161459+0000","last_unstale":"2024-02-21T23:25:22.636614+0000","last_undegraded":"2024-02-21T23:25:22.636614+0000","last_fullsized":"2024-02-21T23:25:22.636614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:31:58.434561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195408+0000","last_change":"2024-02-21T23:25:16.147162+0000","last_active":"2024-02-21T23:25:23.195408+0000","last_peered":"2024-02-21T23:25:23.195408+0000","last_clean":"2024-02-21T23:25:23.195408+0000","last_became_active":"2024-02-21T23:25:16.146850+0000","last_became_peered":"2024-02-21T23:25:16.146850+0000","last_unstale":"2024-02-21T23:25:23.195408+0000","last_undegraded":"2024-02-21T23:25:23.195408+0000","last_fullsized":"2024-02-21T23:25:23.195408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637976+0000","last_change":"2024-02-21T23:25:22.248109+0000","last_active":"2024-02-21T23:25:22.637976+0000","last_peered":"2024-02-21T23:25:22.637976+0000","last_clean":"2024-02-21T23:25:22.637976+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:25:22.637976+0000","last_undegraded":"2024-02-21T23:25:22.637976+0000","last_fullsized":"2024-02-21T23:25:22.637976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194934+0000","last_change":"2024-02-21T23:25:17.192450+0000","last_active":"2024-02-21T23:25:23.194934+0000","last_peered":"2024-02-21T23:25:23.194934+0000","last_clean":"2024-02-21T23:25:23.194934+0000","last_became_active":"2024-02-21T23:25:17.192167+0000","last_became_peered":"2024-02-21T23:25:17.192167+0000","last_unstale":"2024-02-21T23:25:23.194934+0000","last_undegraded":"2024-02-21T23:25:23.194934+0000","last_fullsized":"2024-02-21T23:25:23.194934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197488+0000","last_change":"2024-02-21T23:25:16.127688+0000","last_active":"2024-02-21T23:25:23.197488+0000","last_peered":"2024-02-21T23:25:23.197488+0000","last_clean":"2024-02-21T23:25:23.197488+0000","last_became_active":"2024-02-21T23:25:16.127388+0000","last_became_peered":"2024-02-21T23:25:16.127388+0000","last_unstale":"2024-02-21T23:25:23.197488+0000","last_undegraded":"2024-02-21T23:25:23.197488+0000","last_fullsized":"2024-02-21T23:25:23.197488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195483+0000","last_change":"2024-02-21T23:25:05.612817+0000","last_active":"2024-02-21T23:25:23.195483+0000","last_peered":"2024-02-21T23:25:23.195483+0000","last_clean":"2024-02-21T23:25:23.195483+0000","last_became_active":"2024-02-21T23:25:05.612483+0000","last_became_peered":"2024-02-21T23:25:05.612483+0000","last_unstale":"2024-02-21T23:25:23.195483+0000","last_undegraded":"2024-02-21T23:25:23.195483+0000","last_fullsized":"2024-02-21T23:25:23.195483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_clean_scrub_stamp":"2024-02-21T23:25:03.736732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:23:33.073040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670664+0000","last_change":"2024-02-21T23:25:22.248174+0000","last_active":"2024-02-21T23:25:22.670664+0000","last_peered":"2024-02-21T23:25:22.670664+0000","last_clean":"2024-02-21T23:25:22.670664+0000","last_became_active":"2024-02-21T23:25:22.247834+0000","last_became_peered":"2024-02-21T23:25:22.247834+0000","last_unstale":"2024-02-21T23:25:22.670664+0000","last_undegraded":"2024-02-21T23:25:22.670664+0000","last_fullsized":"2024-02-21T23:25:22.670664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:13.953358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196092+0000","last_change":"2024-02-21T23:25:17.192575+0000","last_active":"2024-02-21T23:25:23.196092+0000","last_peered":"2024-02-21T23:25:23.196092+0000","last_clean":"2024-02-21T23:25:23.196092+0000","last_became_active":"2024-02-21T23:25:17.192293+0000","last_became_peered":"2024-02-21T23:25:17.192293+0000","last_unstale":"2024-02-21T23:25:23.196092+0000","last_undegraded":"2024-02-21T23:25:23.196092+0000","last_fullsized":"2024-02-21T23:25:23.196092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637073+0000","last_change":"2024-02-21T23:25:16.128369+0000","last_active":"2024-02-21T23:25:22.637073+0000","last_peered":"2024-02-21T23:25:22.637073+0000","last_clean":"2024-02-21T23:25:22.637073+0000","last_became_active":"2024-02-21T23:25:16.128067+0000","last_became_peered":"2024-02-21T23:25:16.128067+0000","last_unstale":"2024-02-21T23:25:22.637073+0000","last_undegraded":"2024-02-21T23:25:22.637073+0000","last_fullsized":"2024-02-21T23:25:22.637073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670470+0000","last_change":"2024-02-21T23:25:22.243282+0000","last_active":"2024-02-21T23:25:22.670470+0000","last_peered":"2024-02-21T23:25:22.670470+0000","last_clean":"2024-02-21T23:25:22.670470+0000","last_became_active":"2024-02-21T23:25:22.242916+0000","last_became_peered":"2024-02-21T23:25:22.242916+0000","last_unstale":"2024-02-21T23:25:22.670470+0000","last_undegraded":"2024-02-21T23:25:22.670470+0000","last_fullsized":"2024-02-21T23:25:22.670470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197322+0000","last_change":"2024-02-21T23:25:17.185418+0000","last_active":"2024-02-21T23:25:23.197322+0000","last_peered":"2024-02-21T23:25:23.197322+0000","last_clean":"2024-02-21T23:25:23.197322+0000","last_became_active":"2024-02-21T23:25:17.185119+0000","last_became_peered":"2024-02-21T23:25:17.185119+0000","last_unstale":"2024-02-21T23:25:23.197322+0000","last_undegraded":"2024-02-21T23:25:23.197322+0000","last_fullsized":"2024-02-21T23:25:23.197322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723756+0000","last_change":"2024-02-21T23:25:16.151088+0000","last_active":"2024-02-21T23:25:22.723756+0000","last_peered":"2024-02-21T23:25:22.723756+0000","last_clean":"2024-02-21T23:25:22.723756+0000","last_became_active":"2024-02-21T23:25:16.150819+0000","last_became_peered":"2024-02-21T23:25:16.150819+0000","last_unstale":"2024-02-21T23:25:22.723756+0000","last_undegraded":"2024-02-21T23:25:22.723756+0000","last_fullsized":"2024-02-21T23:25:22.723756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637420+0000","last_change":"2024-02-21T23:25:21.684592+0000","last_active":"2024-02-21T23:25:22.637420+0000","last_peered":"2024-02-21T23:25:22.637420+0000","last_clean":"2024-02-21T23:25:22.637420+0000","last_became_active":"2024-02-21T23:25:21.684192+0000","last_became_peered":"2024-02-21T23:25:21.684192+0000","last_unstale":"2024-02-21T23:25:22.637420+0000","last_undegraded":"2024-02-21T23:25:22.637420+0000","last_fullsized":"2024-02-21T23:25:22.637420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196248+0000","last_change":"2024-02-21T23:25:17.184268+0000","last_active":"2024-02-21T23:25:23.196248+0000","last_peered":"2024-02-21T23:25:23.196248+0000","last_clean":"2024-02-21T23:25:23.196248+0000","last_became_active":"2024-02-21T23:25:17.184027+0000","last_became_peered":"2024-02-21T23:25:17.184027+0000","last_unstale":"2024-02-21T23:25:23.196248+0000","last_undegraded":"2024-02-21T23:25:23.196248+0000","last_fullsized":"2024-02-21T23:25:23.196248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637140+0000","last_change":"2024-02-21T23:25:16.108099+0000","last_active":"2024-02-21T23:25:22.637140+0000","last_peered":"2024-02-21T23:25:22.637140+0000","last_clean":"2024-02-21T23:25:22.637140+0000","last_became_active":"2024-02-21T23:25:16.107815+0000","last_became_peered":"2024-02-21T23:25:16.107815+0000","last_unstale":"2024-02-21T23:25:22.637140+0000","last_undegraded":"2024-02-21T23:25:22.637140+0000","last_fullsized":"2024-02-21T23:25:22.637140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195214+0000","last_change":"2024-02-21T23:25:22.248926+0000","last_active":"2024-02-21T23:25:23.195214+0000","last_peered":"2024-02-21T23:25:23.195214+0000","last_clean":"2024-02-21T23:25:23.195214+0000","last_became_active":"2024-02-21T23:25:22.248558+0000","last_became_peered":"2024-02-21T23:25:22.248558+0000","last_unstale":"2024-02-21T23:25:23.195214+0000","last_undegraded":"2024-02-21T23:25:23.195214+0000","last_fullsized":"2024-02-21T23:25:23.195214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195709+0000","last_change":"2024-02-21T23:25:17.183254+0000","last_active":"2024-02-21T23:25:23.195709+0000","last_peered":"2024-02-21T23:25:23.195709+0000","last_clean":"2024-02-21T23:25:23.195709+0000","last_became_active":"2024-02-21T23:25:17.183020+0000","last_became_peered":"2024-02-21T23:25:17.183020+0000","last_unstale":"2024-02-21T23:25:23.195709+0000","last_undegraded":"2024-02-21T23:25:23.195709+0000","last_fullsized":"2024-02-21T23:25:23.195709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196256+0000","last_change":"2024-02-21T23:25:16.129131+0000","last_active":"2024-02-21T23:25:23.196256+0000","last_peered":"2024-02-21T23:25:23.196256+0000","last_clean":"2024-02-21T23:25:23.196256+0000","last_became_active":"2024-02-21T23:25:16.128853+0000","last_became_peered":"2024-02-21T23:25:16.128853+0000","last_unstale":"2024-02-21T23:25:23.196256+0000","last_undegraded":"2024-02-21T23:25:23.196256+0000","last_fullsized":"2024-02-21T23:25:23.196256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:29.302181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195886+0000","last_change":"2024-02-21T23:25:21.684342+0000","last_active":"2024-02-21T23:25:23.195886+0000","last_peered":"2024-02-21T23:25:23.195886+0000","last_clean":"2024-02-21T23:25:23.195886+0000","last_became_active":"2024-02-21T23:25:21.683991+0000","last_became_peered":"2024-02-21T23:25:21.683991+0000","last_unstale":"2024-02-21T23:25:23.195886+0000","last_undegraded":"2024-02-21T23:25:23.195886+0000","last_fullsized":"2024-02-21T23:25:23.195886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:21:00.312314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195905+0000","last_change":"2024-02-21T23:25:17.183442+0000","last_active":"2024-02-21T23:25:23.195905+0000","last_peered":"2024-02-21T23:25:23.195905+0000","last_clean":"2024-02-21T23:25:23.195905+0000","last_became_active":"2024-02-21T23:25:17.183057+0000","last_became_peered":"2024-02-21T23:25:17.183057+0000","last_unstale":"2024-02-21T23:25:23.195905+0000","last_undegraded":"2024-02-21T23:25:23.195905+0000","last_fullsized":"2024-02-21T23:25:23.195905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636480+0000","last_change":"2024-02-21T23:25:16.145801+0000","last_active":"2024-02-21T23:25:22.636480+0000","last_peered":"2024-02-21T23:25:22.636480+0000","last_clean":"2024-02-21T23:25:22.636480+0000","last_became_active":"2024-02-21T23:25:16.145523+0000","last_became_peered":"2024-02-21T23:25:16.145523+0000","last_unstale":"2024-02-21T23:25:22.636480+0000","last_undegraded":"2024-02-21T23:25:22.636480+0000","last_fullsized":"2024-02-21T23:25:22.636480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637169+0000","last_change":"2024-02-21T23:25:21.679191+0000","last_active":"2024-02-21T23:25:22.637169+0000","last_peered":"2024-02-21T23:25:22.637169+0000","last_clean":"2024-02-21T23:25:22.637169+0000","last_became_active":"2024-02-21T23:25:21.678715+0000","last_became_peered":"2024-02-21T23:25:21.678715+0000","last_unstale":"2024-02-21T23:25:22.637169+0000","last_undegraded":"2024-02-21T23:25:22.637169+0000","last_fullsized":"2024-02-21T23:25:22.637169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670230+0000","last_change":"2024-02-21T23:25:17.138401+0000","last_active":"2024-02-21T23:25:22.670230+0000","last_peered":"2024-02-21T23:25:22.670230+0000","last_clean":"2024-02-21T23:25:22.670230+0000","last_became_active":"2024-02-21T23:25:17.138136+0000","last_became_peered":"2024-02-21T23:25:17.138136+0000","last_unstale":"2024-02-21T23:25:22.670230+0000","last_undegraded":"2024-02-21T23:25:22.670230+0000","last_fullsized":"2024-02-21T23:25:22.670230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670228+0000","last_change":"2024-02-21T23:25:16.130209+0000","last_active":"2024-02-21T23:25:22.670228+0000","last_peered":"2024-02-21T23:25:22.670228+0000","last_clean":"2024-02-21T23:25:22.670228+0000","last_became_active":"2024-02-21T23:25:16.129974+0000","last_became_peered":"2024-02-21T23:25:16.129974+0000","last_unstale":"2024-02-21T23:25:22.670228+0000","last_undegraded":"2024-02-21T23:25:22.670228+0000","last_fullsized":"2024-02-21T23:25:22.670228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195559+0000","last_change":"2024-02-21T23:25:22.243303+0000","last_active":"2024-02-21T23:25:23.195559+0000","last_peered":"2024-02-21T23:25:23.195559+0000","last_clean":"2024-02-21T23:25:23.195559+0000","last_became_active":"2024-02-21T23:25:22.242960+0000","last_became_peered":"2024-02-21T23:25:22.242960+0000","last_unstale":"2024-02-21T23:25:23.195559+0000","last_undegraded":"2024-02-21T23:25:23.195559+0000","last_fullsized":"2024-02-21T23:25:23.195559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:54:16.386354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196403+0000","last_change":"2024-02-21T23:25:17.184233+0000","last_active":"2024-02-21T23:25:23.196403+0000","last_peered":"2024-02-21T23:25:23.196403+0000","last_clean":"2024-02-21T23:25:23.196403+0000","last_became_active":"2024-02-21T23:25:17.183980+0000","last_became_peered":"2024-02-21T23:25:17.183980+0000","last_unstale":"2024-02-21T23:25:23.196403+0000","last_undegraded":"2024-02-21T23:25:23.196403+0000","last_fullsized":"2024-02-21T23:25:23.196403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197566+0000","last_change":"2024-02-21T23:25:16.131147+0000","last_active":"2024-02-21T23:25:23.197566+0000","last_peered":"2024-02-21T23:25:23.197566+0000","last_clean":"2024-02-21T23:25:23.197566+0000","last_became_active":"2024-02-21T23:25:16.130553+0000","last_became_peered":"2024-02-21T23:25:16.130553+0000","last_unstale":"2024-02-21T23:25:23.197566+0000","last_undegraded":"2024-02-21T23:25:23.197566+0000","last_fullsized":"2024-02-21T23:25:23.197566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195579+0000","last_change":"2024-02-21T23:25:22.243261+0000","last_active":"2024-02-21T23:25:23.195579+0000","last_peered":"2024-02-21T23:25:23.195579+0000","last_clean":"2024-02-21T23:25:23.195579+0000","last_became_active":"2024-02-21T23:25:22.242900+0000","last_became_peered":"2024-02-21T23:25:22.242900+0000","last_unstale":"2024-02-21T23:25:23.195579+0000","last_undegraded":"2024-02-21T23:25:23.195579+0000","last_fullsized":"2024-02-21T23:25:23.195579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:34:44.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637437+0000","last_change":"2024-02-21T23:25:17.183303+0000","last_active":"2024-02-21T23:25:22.637437+0000","last_peered":"2024-02-21T23:25:22.637437+0000","last_clean":"2024-02-21T23:25:22.637437+0000","last_became_active":"2024-02-21T23:25:17.183042+0000","last_became_peered":"2024-02-21T23:25:17.183042+0000","last_unstale":"2024-02-21T23:25:22.637437+0000","last_undegraded":"2024-02-21T23:25:22.637437+0000","last_fullsized":"2024-02-21T23:25:22.637437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637466+0000","last_change":"2024-02-21T23:25:16.127609+0000","last_active":"2024-02-21T23:25:22.637466+0000","last_peered":"2024-02-21T23:25:22.637466+0000","last_clean":"2024-02-21T23:25:22.637466+0000","last_became_active":"2024-02-21T23:25:16.127375+0000","last_became_peered":"2024-02-21T23:25:16.127375+0000","last_unstale":"2024-02-21T23:25:22.637466+0000","last_undegraded":"2024-02-21T23:25:22.637466+0000","last_fullsized":"2024-02-21T23:25:22.637466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197762+0000","last_change":"2024-02-21T23:25:22.248329+0000","last_active":"2024-02-21T23:25:23.197762+0000","last_peered":"2024-02-21T23:25:23.197762+0000","last_clean":"2024-02-21T23:25:23.197762+0000","last_became_active":"2024-02-21T23:25:22.247889+0000","last_became_peered":"2024-02-21T23:25:22.247889+0000","last_unstale":"2024-02-21T23:25:23.197762+0000","last_undegraded":"2024-02-21T23:25:23.197762+0000","last_fullsized":"2024-02-21T23:25:23.197762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196057+0000","last_change":"2024-02-21T23:25:17.183741+0000","last_active":"2024-02-21T23:25:23.196057+0000","last_peered":"2024-02-21T23:25:23.196057+0000","last_clean":"2024-02-21T23:25:23.196057+0000","last_became_active":"2024-02-21T23:25:17.183408+0000","last_became_peered":"2024-02-21T23:25:17.183408+0000","last_unstale":"2024-02-21T23:25:23.196057+0000","last_undegraded":"2024-02-21T23:25:23.196057+0000","last_fullsized":"2024-02-21T23:25:23.196057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:38.383407+0000","last_change":"2024-02-21T23:25:16.116460+0000","last_active":"2024-02-21T23:25:38.383407+0000","last_peered":"2024-02-21T23:25:38.383407+0000","last_clean":"2024-02-21T23:25:38.383407+0000","last_became_active":"2024-02-21T23:25:16.116162+0000","last_became_peered":"2024-02-21T23:25:16.116162+0000","last_unstale":"2024-02-21T23:25:38.383407+0000","last_undegraded":"2024-02-21T23:25:38.383407+0000","last_fullsized":"2024-02-21T23:25:38.383407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723404+0000","last_change":"2024-02-21T23:25:21.683199+0000","last_active":"2024-02-21T23:25:22.723404+0000","last_peered":"2024-02-21T23:25:22.723404+0000","last_clean":"2024-02-21T23:25:22.723404+0000","last_became_active":"2024-02-21T23:25:21.682425+0000","last_became_peered":"2024-02-21T23:25:21.682425+0000","last_unstale":"2024-02-21T23:25:22.723404+0000","last_undegraded":"2024-02-21T23:25:22.723404+0000","last_fullsized":"2024-02-21T23:25:22.723404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196728+0000","last_change":"2024-02-21T23:25:17.184837+0000","last_active":"2024-02-21T23:25:23.196728+0000","last_peered":"2024-02-21T23:25:23.196728+0000","last_clean":"2024-02-21T23:25:23.196728+0000","last_became_active":"2024-02-21T23:25:17.184574+0000","last_became_peered":"2024-02-21T23:25:17.184574+0000","last_unstale":"2024-02-21T23:25:23.196728+0000","last_undegraded":"2024-02-21T23:25:23.196728+0000","last_fullsized":"2024-02-21T23:25:23.196728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723604+0000","last_change":"2024-02-21T23:25:16.151031+0000","last_active":"2024-02-21T23:25:22.723604+0000","last_peered":"2024-02-21T23:25:22.723604+0000","last_clean":"2024-02-21T23:25:22.723604+0000","last_became_active":"2024-02-21T23:25:16.150767+0000","last_became_peered":"2024-02-21T23:25:16.150767+0000","last_unstale":"2024-02-21T23:25:22.723604+0000","last_undegraded":"2024-02-21T23:25:22.723604+0000","last_fullsized":"2024-02-21T23:25:22.723604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196772+0000","last_change":"2024-02-21T23:25:21.683995+0000","last_active":"2024-02-21T23:25:23.196772+0000","last_peered":"2024-02-21T23:25:23.196772+0000","last_clean":"2024-02-21T23:25:23.196772+0000","last_became_active":"2024-02-21T23:25:21.683415+0000","last_became_peered":"2024-02-21T23:25:21.683415+0000","last_unstale":"2024-02-21T23:25:23.196772+0000","last_undegraded":"2024-02-21T23:25:23.196772+0000","last_fullsized":"2024-02-21T23:25:23.196772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637736+0000","last_change":"2024-02-21T23:25:17.187230+0000","last_active":"2024-02-21T23:25:22.637736+0000","last_peered":"2024-02-21T23:25:22.637736+0000","last_clean":"2024-02-21T23:25:22.637736+0000","last_became_active":"2024-02-21T23:25:17.186952+0000","last_became_peered":"2024-02-21T23:25:17.186952+0000","last_unstale":"2024-02-21T23:25:22.637736+0000","last_undegraded":"2024-02-21T23:25:22.637736+0000","last_fullsized":"2024-02-21T23:25:22.637736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197393+0000","last_change":"2024-02-21T23:25:16.134722+0000","last_active":"2024-02-21T23:25:23.197393+0000","last_peered":"2024-02-21T23:25:23.197393+0000","last_clean":"2024-02-21T23:25:23.197393+0000","last_became_active":"2024-02-21T23:25:16.134458+0000","last_became_peered":"2024-02-21T23:25:16.134458+0000","last_unstale":"2024-02-21T23:25:23.197393+0000","last_undegraded":"2024-02-21T23:25:23.197393+0000","last_fullsized":"2024-02-21T23:25:23.197393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195006+0000","last_change":"2024-02-21T23:25:21.684109+0000","last_active":"2024-02-21T23:25:23.195006+0000","last_peered":"2024-02-21T23:25:23.195006+0000","last_clean":"2024-02-21T23:25:23.195006+0000","last_became_active":"2024-02-21T23:25:21.683705+0000","last_became_peered":"2024-02-21T23:25:21.683705+0000","last_unstale":"2024-02-21T23:25:23.195006+0000","last_undegraded":"2024-02-21T23:25:23.195006+0000","last_fullsized":"2024-02-21T23:25:23.195006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197237+0000","last_change":"2024-02-21T23:25:17.188374+0000","last_active":"2024-02-21T23:25:23.197237+0000","last_peered":"2024-02-21T23:25:23.197237+0000","last_clean":"2024-02-21T23:25:23.197237+0000","last_became_active":"2024-02-21T23:25:17.187792+0000","last_became_peered":"2024-02-21T23:25:17.187792+0000","last_unstale":"2024-02-21T23:25:23.197237+0000","last_undegraded":"2024-02-21T23:25:23.197237+0000","last_fullsized":"2024-02-21T23:25:23.197237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670904+0000","last_change":"2024-02-21T23:25:16.129446+0000","last_active":"2024-02-21T23:25:22.670904+0000","last_peered":"2024-02-21T23:25:22.670904+0000","last_clean":"2024-02-21T23:25:22.670904+0000","last_became_active":"2024-02-21T23:25:16.129164+0000","last_became_peered":"2024-02-21T23:25:16.129164+0000","last_unstale":"2024-02-21T23:25:22.670904+0000","last_undegraded":"2024-02-21T23:25:22.670904+0000","last_fullsized":"2024-02-21T23:25:22.670904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194890+0000","last_change":"2024-02-21T23:25:22.248096+0000","last_active":"2024-02-21T23:25:23.194890+0000","last_peered":"2024-02-21T23:25:23.194890+0000","last_clean":"2024-02-21T23:25:23.194890+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:25:23.194890+0000","last_undegraded":"2024-02-21T23:25:23.194890+0000","last_fullsized":"2024-02-21T23:25:23.194890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637006+0000","last_change":"2024-02-21T23:25:22.248119+0000","last_active":"2024-02-21T23:25:22.637006+0000","last_peered":"2024-02-21T23:25:22.637006+0000","last_clean":"2024-02-21T23:25:22.637006+0000","last_became_active":"2024-02-21T23:25:22.247811+0000","last_became_peered":"2024-02-21T23:25:22.247811+0000","last_unstale":"2024-02-21T23:25:22.637006+0000","last_undegraded":"2024-02-21T23:25:22.637006+0000","last_fullsized":"2024-02-21T23:25:22.637006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723036+0000","last_change":"2024-02-21T23:25:17.189176+0000","last_active":"2024-02-21T23:25:22.723036+0000","last_peered":"2024-02-21T23:25:22.723036+0000","last_clean":"2024-02-21T23:25:22.723036+0000","last_became_active":"2024-02-21T23:25:17.188885+0000","last_became_peered":"2024-02-21T23:25:17.188885+0000","last_unstale":"2024-02-21T23:25:22.723036+0000","last_undegraded":"2024-02-21T23:25:22.723036+0000","last_fullsized":"2024-02-21T23:25:22.723036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195502+0000","last_change":"2024-02-21T23:25:16.132036+0000","last_active":"2024-02-21T23:25:23.195502+0000","last_peered":"2024-02-21T23:25:23.195502+0000","last_clean":"2024-02-21T23:25:23.195502+0000","last_became_active":"2024-02-21T23:25:16.131756+0000","last_became_peered":"2024-02-21T23:25:16.131756+0000","last_unstale":"2024-02-21T23:25:23.195502+0000","last_undegraded":"2024-02-21T23:25:23.195502+0000","last_fullsized":"2024-02-21T23:25:23.195502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670902+0000","last_change":"2024-02-21T23:25:21.686133+0000","last_active":"2024-02-21T23:25:22.670902+0000","last_peered":"2024-02-21T23:25:22.670902+0000","last_clean":"2024-02-21T23:25:22.670902+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:25:22.670902+0000","last_undegraded":"2024-02-21T23:25:22.670902+0000","last_fullsized":"2024-02-21T23:25:22.670902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194875+0000","last_change":"2024-02-21T23:25:17.135158+0000","last_active":"2024-02-21T23:25:23.194875+0000","last_peered":"2024-02-21T23:25:23.194875+0000","last_clean":"2024-02-21T23:25:23.194875+0000","last_became_active":"2024-02-21T23:25:17.134888+0000","last_became_peered":"2024-02-21T23:25:17.134888+0000","last_unstale":"2024-02-21T23:25:23.194875+0000","last_undegraded":"2024-02-21T23:25:23.194875+0000","last_fullsized":"2024-02-21T23:25:23.194875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194948+0000","last_change":"2024-02-21T23:25:16.116378+0000","last_active":"2024-02-21T23:25:23.194948+0000","last_peered":"2024-02-21T23:25:23.194948+0000","last_clean":"2024-02-21T23:25:23.194948+0000","last_became_active":"2024-02-21T23:25:16.116201+0000","last_became_peered":"2024-02-21T23:25:16.116201+0000","last_unstale":"2024-02-21T23:25:23.194948+0000","last_undegraded":"2024-02-21T23:25:23.194948+0000","last_fullsized":"2024-02-21T23:25:23.194948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670541+0000","last_change":"2024-02-21T23:25:16.132314+0000","last_active":"2024-02-21T23:25:22.670541+0000","last_peered":"2024-02-21T23:25:22.670541+0000","last_clean":"2024-02-21T23:25:22.670541+0000","last_became_active":"2024-02-21T23:25:16.132062+0000","last_became_peered":"2024-02-21T23:25:16.132062+0000","last_unstale":"2024-02-21T23:25:22.670541+0000","last_undegraded":"2024-02-21T23:25:22.670541+0000","last_fullsized":"2024-02-21T23:25:22.670541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637458+0000","last_change":"2024-02-21T23:25:17.161451+0000","last_active":"2024-02-21T23:25:22.637458+0000","last_peered":"2024-02-21T23:25:22.637458+0000","last_clean":"2024-02-21T23:25:22.637458+0000","last_became_active":"2024-02-21T23:25:17.161176+0000","last_became_peered":"2024-02-21T23:25:17.161176+0000","last_unstale":"2024-02-21T23:25:22.637458+0000","last_undegraded":"2024-02-21T23:25:22.637458+0000","last_fullsized":"2024-02-21T23:25:22.637458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636872+0000","last_change":"2024-02-21T23:25:21.685724+0000","last_active":"2024-02-21T23:25:22.636872+0000","last_peered":"2024-02-21T23:25:22.636872+0000","last_clean":"2024-02-21T23:25:22.636872+0000","last_became_active":"2024-02-21T23:25:21.685375+0000","last_became_peered":"2024-02-21T23:25:21.685375+0000","last_unstale":"2024-02-21T23:25:22.636872+0000","last_undegraded":"2024-02-21T23:25:22.636872+0000","last_fullsized":"2024-02-21T23:25:22.636872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:56:28.795401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194995+0000","last_change":"2024-02-21T23:25:16.134127+0000","last_active":"2024-02-21T23:25:23.194995+0000","last_peered":"2024-02-21T23:25:23.194995+0000","last_clean":"2024-02-21T23:25:23.194995+0000","last_became_active":"2024-02-21T23:25:16.133839+0000","last_became_peered":"2024-02-21T23:25:16.133839+0000","last_unstale":"2024-02-21T23:25:23.194995+0000","last_undegraded":"2024-02-21T23:25:23.194995+0000","last_fullsized":"2024-02-21T23:25:23.194995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195317+0000","last_change":"2024-02-21T23:25:17.190455+0000","last_active":"2024-02-21T23:25:23.195317+0000","last_peered":"2024-02-21T23:25:23.195317+0000","last_clean":"2024-02-21T23:25:23.195317+0000","last_became_active":"2024-02-21T23:25:17.190194+0000","last_became_peered":"2024-02-21T23:25:17.190194+0000","last_unstale":"2024-02-21T23:25:23.195317+0000","last_undegraded":"2024-02-21T23:25:23.195317+0000","last_fullsized":"2024-02-21T23:25:23.195317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:03.451409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195261+0000","last_change":"2024-02-21T23:25:21.683501+0000","last_active":"2024-02-21T23:25:23.195261+0000","last_peered":"2024-02-21T23:25:23.195261+0000","last_clean":"2024-02-21T23:25:23.195261+0000","last_became_active":"2024-02-21T23:25:21.683139+0000","last_became_peered":"2024-02-21T23:25:21.683139+0000","last_unstale":"2024-02-21T23:25:23.195261+0000","last_undegraded":"2024-02-21T23:25:23.195261+0000","last_fullsized":"2024-02-21T23:25:23.195261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-22T23:58:15.072875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670359+0000","last_change":"2024-02-21T23:25:17.138039+0000","last_active":"2024-02-21T23:25:22.670359+0000","last_peered":"2024-02-21T23:25:22.670359+0000","last_clean":"2024-02-21T23:25:22.670359+0000","last_became_active":"2024-02-21T23:25:17.137717+0000","last_became_peered":"2024-02-21T23:25:17.137717+0000","last_unstale":"2024-02-21T23:25:22.670359+0000","last_undegraded":"2024-02-21T23:25:22.670359+0000","last_fullsized":"2024-02-21T23:25:22.670359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.636758+0000","last_change":"2024-02-21T23:25:17.194727+0000","last_active":"2024-02-21T23:25:22.636758+0000","last_peered":"2024-02-21T23:25:22.636758+0000","last_clean":"2024-02-21T23:25:22.636758+0000","last_became_active":"2024-02-21T23:25:17.194533+0000","last_became_peered":"2024-02-21T23:25:17.194533+0000","last_unstale":"2024-02-21T23:25:22.636758+0000","last_undegraded":"2024-02-21T23:25:22.636758+0000","last_fullsized":"2024-02-21T23:25:22.636758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197507+0000","last_change":"2024-02-21T23:25:17.191051+0000","last_active":"2024-02-21T23:25:23.197507+0000","last_peered":"2024-02-21T23:25:23.197507+0000","last_clean":"2024-02-21T23:25:23.197507+0000","last_became_active":"2024-02-21T23:25:17.190600+0000","last_became_peered":"2024-02-21T23:25:17.190600+0000","last_unstale":"2024-02-21T23:25:23.197507+0000","last_undegraded":"2024-02-21T23:25:23.197507+0000","last_fullsized":"2024-02-21T23:25:23.197507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637377+0000","last_change":"2024-02-21T23:25:17.135311+0000","last_active":"2024-02-21T23:25:22.637377+0000","last_peered":"2024-02-21T23:25:22.637377+0000","last_clean":"2024-02-21T23:25:22.637377+0000","last_became_active":"2024-02-21T23:25:17.135011+0000","last_became_peered":"2024-02-21T23:25:17.135011+0000","last_unstale":"2024-02-21T23:25:22.637377+0000","last_undegraded":"2024-02-21T23:25:22.637377+0000","last_fullsized":"2024-02-21T23:25:22.637377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195670+0000","last_change":"2024-02-21T23:25:17.161696+0000","last_active":"2024-02-21T23:25:23.195670+0000","last_peered":"2024-02-21T23:25:23.195670+0000","last_clean":"2024-02-21T23:25:23.195670+0000","last_became_active":"2024-02-21T23:25:17.161401+0000","last_became_peered":"2024-02-21T23:25:17.161401+0000","last_unstale":"2024-02-21T23:25:23.195670+0000","last_undegraded":"2024-02-21T23:25:23.195670+0000","last_fullsized":"2024-02-21T23:25:23.195670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723507+0000","last_change":"2024-02-21T23:25:17.189731+0000","last_active":"2024-02-21T23:25:22.723507+0000","last_peered":"2024-02-21T23:25:22.723507+0000","last_clean":"2024-02-21T23:25:22.723507+0000","last_became_active":"2024-02-21T23:25:17.189484+0000","last_became_peered":"2024-02-21T23:25:17.189484+0000","last_unstale":"2024-02-21T23:25:22.723507+0000","last_undegraded":"2024-02-21T23:25:22.723507+0000","last_fullsized":"2024-02-21T23:25:22.723507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670963+0000","last_change":"2024-02-21T23:25:17.184125+0000","last_active":"2024-02-21T23:25:22.670963+0000","last_peered":"2024-02-21T23:25:22.670963+0000","last_clean":"2024-02-21T23:25:22.670963+0000","last_became_active":"2024-02-21T23:25:17.183865+0000","last_became_peered":"2024-02-21T23:25:17.183865+0000","last_unstale":"2024-02-21T23:25:22.670963+0000","last_undegraded":"2024-02-21T23:25:22.670963+0000","last_fullsized":"2024-02-21T23:25:22.670963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195515+0000","last_change":"2024-02-21T23:25:17.161482+0000","last_active":"2024-02-21T23:25:23.195515+0000","last_peered":"2024-02-21T23:25:23.195515+0000","last_clean":"2024-02-21T23:25:23.195515+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:25:23.195515+0000","last_undegraded":"2024-02-21T23:25:23.195515+0000","last_fullsized":"2024-02-21T23:25:23.195515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195414+0000","last_change":"2024-02-21T23:25:17.185691+0000","last_active":"2024-02-21T23:25:23.195414+0000","last_peered":"2024-02-21T23:25:23.195414+0000","last_clean":"2024-02-21T23:25:23.195414+0000","last_became_active":"2024-02-21T23:25:17.185420+0000","last_became_peered":"2024-02-21T23:25:17.185420+0000","last_unstale":"2024-02-21T23:25:23.195414+0000","last_undegraded":"2024-02-21T23:25:23.195414+0000","last_fullsized":"2024-02-21T23:25:23.195414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194975+0000","last_change":"2024-02-21T23:25:17.161321+0000","last_active":"2024-02-21T23:25:23.194975+0000","last_peered":"2024-02-21T23:25:23.194975+0000","last_clean":"2024-02-21T23:25:23.194975+0000","last_became_active":"2024-02-21T23:25:17.161010+0000","last_became_peered":"2024-02-21T23:25:17.161010+0000","last_unstale":"2024-02-21T23:25:23.194975+0000","last_undegraded":"2024-02-21T23:25:23.194975+0000","last_fullsized":"2024-02-21T23:25:23.194975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197503+0000","last_change":"2024-02-21T23:25:17.191072+0000","last_active":"2024-02-21T23:25:23.197503+0000","last_peered":"2024-02-21T23:25:23.197503+0000","last_clean":"2024-02-21T23:25:23.197503+0000","last_became_active":"2024-02-21T23:25:17.187741+0000","last_became_peered":"2024-02-21T23:25:17.187741+0000","last_unstale":"2024-02-21T23:25:23.197503+0000","last_undegraded":"2024-02-21T23:25:23.197503+0000","last_fullsized":"2024-02-21T23:25:23.197503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637229+0000","last_change":"2024-02-21T23:25:17.186616+0000","last_active":"2024-02-21T23:25:22.637229+0000","last_peered":"2024-02-21T23:25:22.637229+0000","last_clean":"2024-02-21T23:25:22.637229+0000","last_became_active":"2024-02-21T23:25:17.186108+0000","last_became_peered":"2024-02-21T23:25:17.186108+0000","last_unstale":"2024-02-21T23:25:22.637229+0000","last_undegraded":"2024-02-21T23:25:22.637229+0000","last_fullsized":"2024-02-21T23:25:22.637229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670481+0000","last_change":"2024-02-21T23:25:17.161428+0000","last_active":"2024-02-21T23:25:22.670481+0000","last_peered":"2024-02-21T23:25:22.670481+0000","last_clean":"2024-02-21T23:25:22.670481+0000","last_became_active":"2024-02-21T23:25:17.161140+0000","last_became_peered":"2024-02-21T23:25:17.161140+0000","last_unstale":"2024-02-21T23:25:22.670481+0000","last_undegraded":"2024-02-21T23:25:22.670481+0000","last_fullsized":"2024-02-21T23:25:22.670481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196229+0000","last_change":"2024-02-21T23:25:17.190666+0000","last_active":"2024-02-21T23:25:23.196229+0000","last_peered":"2024-02-21T23:25:23.196229+0000","last_clean":"2024-02-21T23:25:23.196229+0000","last_became_active":"2024-02-21T23:25:17.190420+0000","last_became_peered":"2024-02-21T23:25:17.190420+0000","last_unstale":"2024-02-21T23:25:23.196229+0000","last_undegraded":"2024-02-21T23:25:23.196229+0000","last_fullsized":"2024-02-21T23:25:23.196229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637103+0000","last_change":"2024-02-21T23:25:17.189583+0000","last_active":"2024-02-21T23:25:22.637103+0000","last_peered":"2024-02-21T23:25:22.637103+0000","last_clean":"2024-02-21T23:25:22.637103+0000","last_became_active":"2024-02-21T23:25:17.189302+0000","last_became_peered":"2024-02-21T23:25:17.189302+0000","last_unstale":"2024-02-21T23:25:22.637103+0000","last_undegraded":"2024-02-21T23:25:22.637103+0000","last_fullsized":"2024-02-21T23:25:22.637103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197389+0000","last_change":"2024-02-21T23:25:17.188113+0000","last_active":"2024-02-21T23:25:23.197389+0000","last_peered":"2024-02-21T23:25:23.197389+0000","last_clean":"2024-02-21T23:25:23.197389+0000","last_became_active":"2024-02-21T23:25:17.187811+0000","last_became_peered":"2024-02-21T23:25:17.187811+0000","last_unstale":"2024-02-21T23:25:23.197389+0000","last_undegraded":"2024-02-21T23:25:23.197389+0000","last_fullsized":"2024-02-21T23:25:23.197389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195329+0000","last_change":"2024-02-21T23:25:17.186415+0000","last_active":"2024-02-21T23:25:23.195329+0000","last_peered":"2024-02-21T23:25:23.195329+0000","last_clean":"2024-02-21T23:25:23.195329+0000","last_became_active":"2024-02-21T23:25:17.186021+0000","last_became_peered":"2024-02-21T23:25:17.186021+0000","last_unstale":"2024-02-21T23:25:23.195329+0000","last_undegraded":"2024-02-21T23:25:23.195329+0000","last_fullsized":"2024-02-21T23:25:23.195329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723114+0000","last_change":"2024-02-21T23:25:17.186378+0000","last_active":"2024-02-21T23:25:22.723114+0000","last_peered":"2024-02-21T23:25:22.723114+0000","last_clean":"2024-02-21T23:25:22.723114+0000","last_became_active":"2024-02-21T23:25:17.186086+0000","last_became_peered":"2024-02-21T23:25:17.186086+0000","last_unstale":"2024-02-21T23:25:22.723114+0000","last_undegraded":"2024-02-21T23:25:22.723114+0000","last_fullsized":"2024-02-21T23:25:22.723114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637345+0000","last_change":"2024-02-21T23:25:17.190934+0000","last_active":"2024-02-21T23:25:22.637345+0000","last_peered":"2024-02-21T23:25:22.637345+0000","last_clean":"2024-02-21T23:25:22.637345+0000","last_became_active":"2024-02-21T23:25:17.190680+0000","last_became_peered":"2024-02-21T23:25:17.190680+0000","last_unstale":"2024-02-21T23:25:22.637345+0000","last_undegraded":"2024-02-21T23:25:22.637345+0000","last_fullsized":"2024-02-21T23:25:22.637345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:22:04.238374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194866+0000","last_change":"2024-02-21T23:25:17.135140+0000","last_active":"2024-02-21T23:25:23.194866+0000","last_peered":"2024-02-21T23:25:23.194866+0000","last_clean":"2024-02-21T23:25:23.194866+0000","last_became_active":"2024-02-21T23:25:17.134866+0000","last_became_peered":"2024-02-21T23:25:17.134866+0000","last_unstale":"2024-02-21T23:25:23.194866+0000","last_undegraded":"2024-02-21T23:25:23.194866+0000","last_fullsized":"2024-02-21T23:25:23.194866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196657+0000","last_change":"2024-02-21T23:25:17.162204+0000","last_active":"2024-02-21T23:25:23.196657+0000","last_peered":"2024-02-21T23:25:23.196657+0000","last_clean":"2024-02-21T23:25:23.196657+0000","last_became_active":"2024-02-21T23:25:17.161807+0000","last_became_peered":"2024-02-21T23:25:17.161807+0000","last_unstale":"2024-02-21T23:25:23.196657+0000","last_undegraded":"2024-02-21T23:25:23.196657+0000","last_fullsized":"2024-02-21T23:25:23.196657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670677+0000","last_change":"2024-02-21T23:25:17.188932+0000","last_active":"2024-02-21T23:25:22.670677+0000","last_peered":"2024-02-21T23:25:22.670677+0000","last_clean":"2024-02-21T23:25:22.670677+0000","last_became_active":"2024-02-21T23:25:17.188646+0000","last_became_peered":"2024-02-21T23:25:17.188646+0000","last_unstale":"2024-02-21T23:25:22.670677+0000","last_undegraded":"2024-02-21T23:25:22.670677+0000","last_fullsized":"2024-02-21T23:25:22.670677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.670892+0000","last_change":"2024-02-21T23:25:17.188333+0000","last_active":"2024-02-21T23:25:22.670892+0000","last_peered":"2024-02-21T23:25:22.670892+0000","last_clean":"2024-02-21T23:25:22.670892+0000","last_became_active":"2024-02-21T23:25:17.188066+0000","last_became_peered":"2024-02-21T23:25:17.188066+0000","last_unstale":"2024-02-21T23:25:22.670892+0000","last_undegraded":"2024-02-21T23:25:22.670892+0000","last_fullsized":"2024-02-21T23:25:22.670892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196168+0000","last_change":"2024-02-21T23:25:17.187549+0000","last_active":"2024-02-21T23:25:23.196168+0000","last_peered":"2024-02-21T23:25:23.196168+0000","last_clean":"2024-02-21T23:25:23.196168+0000","last_became_active":"2024-02-21T23:25:17.187297+0000","last_became_peered":"2024-02-21T23:25:17.187297+0000","last_unstale":"2024-02-21T23:25:23.196168+0000","last_undegraded":"2024-02-21T23:25:23.196168+0000","last_fullsized":"2024-02-21T23:25:23.196168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.194774+0000","last_change":"2024-02-21T23:25:17.187352+0000","last_active":"2024-02-21T23:25:23.194774+0000","last_peered":"2024-02-21T23:25:23.194774+0000","last_clean":"2024-02-21T23:25:23.194774+0000","last_became_active":"2024-02-21T23:25:17.187156+0000","last_became_peered":"2024-02-21T23:25:17.187156+0000","last_unstale":"2024-02-21T23:25:23.194774+0000","last_undegraded":"2024-02-21T23:25:23.194774+0000","last_fullsized":"2024-02-21T23:25:23.194774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.195829+0000","last_change":"2024-02-21T23:25:17.183264+0000","last_active":"2024-02-21T23:25:23.195829+0000","last_peered":"2024-02-21T23:25:23.195829+0000","last_clean":"2024-02-21T23:25:23.195829+0000","last_became_active":"2024-02-21T23:25:17.182924+0000","last_became_peered":"2024-02-21T23:25:17.182924+0000","last_unstale":"2024-02-21T23:25:23.195829+0000","last_undegraded":"2024-02-21T23:25:23.195829+0000","last_fullsized":"2024-02-21T23:25:23.195829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197389+0000","last_change":"2024-02-21T23:25:17.190841+0000","last_active":"2024-02-21T23:25:23.197389+0000","last_peered":"2024-02-21T23:25:23.197389+0000","last_clean":"2024-02-21T23:25:23.197389+0000","last_became_active":"2024-02-21T23:25:17.190276+0000","last_became_peered":"2024-02-21T23:25:17.190276+0000","last_unstale":"2024-02-21T23:25:23.197389+0000","last_undegraded":"2024-02-21T23:25:23.197389+0000","last_fullsized":"2024-02-21T23:25:23.197389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196058+0000","last_change":"2024-02-21T23:25:17.189872+0000","last_active":"2024-02-21T23:25:23.196058+0000","last_peered":"2024-02-21T23:25:23.196058+0000","last_clean":"2024-02-21T23:25:23.196058+0000","last_became_active":"2024-02-21T23:25:17.189614+0000","last_became_peered":"2024-02-21T23:25:17.189614+0000","last_unstale":"2024-02-21T23:25:23.196058+0000","last_undegraded":"2024-02-21T23:25:23.196058+0000","last_fullsized":"2024-02-21T23:25:23.196058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.197136+0000","last_change":"2024-02-21T23:25:17.185771+0000","last_active":"2024-02-21T23:25:23.197136+0000","last_peered":"2024-02-21T23:25:23.197136+0000","last_clean":"2024-02-21T23:25:23.197136+0000","last_became_active":"2024-02-21T23:25:17.185458+0000","last_became_peered":"2024-02-21T23:25:17.185458+0000","last_unstale":"2024-02-21T23:25:23.197136+0000","last_undegraded":"2024-02-21T23:25:23.197136+0000","last_fullsized":"2024-02-21T23:25:23.197136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.637050+0000","last_change":"2024-02-21T23:25:17.191159+0000","last_active":"2024-02-21T23:25:22.637050+0000","last_peered":"2024-02-21T23:25:22.637050+0000","last_clean":"2024-02-21T23:25:22.637050+0000","last_became_active":"2024-02-21T23:25:17.190576+0000","last_became_peered":"2024-02-21T23:25:17.190576+0000","last_unstale":"2024-02-21T23:25:22.637050+0000","last_undegraded":"2024-02-21T23:25:22.637050+0000","last_fullsized":"2024-02-21T23:25:22.637050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:22.723892+0000","last_change":"2024-02-21T23:25:17.189287+0000","last_active":"2024-02-21T23:25:22.723892+0000","last_peered":"2024-02-21T23:25:22.723892+0000","last_clean":"2024-02-21T23:25:22.723892+0000","last_became_active":"2024-02-21T23:25:17.189029+0000","last_became_peered":"2024-02-21T23:25:17.189029+0000","last_unstale":"2024-02-21T23:25:22.723892+0000","last_undegraded":"2024-02-21T23:25:22.723892+0000","last_fullsized":"2024-02-21T23:25:22.723892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-21T23:25:23.196058+0000","last_change":"2024-02-21T23:25:17.183737+0000","last_active":"2024-02-21T23:25:23.196058+0000","last_peered":"2024-02-21T23:25:23.196058+0000","last_clean":"2024-02-21T23:25:23.196058+0000","last_became_active":"2024-02-21T23:25:17.183410+0000","last_became_peered":"2024-02-21T23:25:17.183410+0000","last_unstale":"2024-02-21T23:25:23.196058+0000","last_undegraded":"2024-02-21T23:25:23.196058+0000","last_fullsized":"2024-02-21T23:25:23.196058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":691206,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":691262,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-21T23:25:58.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-21T23:25:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.387+0000 7f912ed1e640 1 Processor -- start 2024-02-21T23:25:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.387+0000 7f912ed1e640 1 -- start start 2024-02-21T23:25:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.388+0000 7f912ed1e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f912815bec0 0x7f9128071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.388+0000 7f912ed1e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9128158230 con 0x7f91280c53a0 2024-02-21T23:25:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.388+0000 7f912ed1e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9128158510 con 0x7f91280c4fc0 2024-02-21T23:25:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.388+0000 7f912ed1e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9128158eb0 con 0x7f912815bec0 2024-02-21T23:25:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.389+0000 7f9127fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f91280c53a0 0x7f912815b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51452/0 (socket says 172.21.15.133:51452) 2024-02-21T23:25:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.389+0000 7f912d2bb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f912815bec0 0x7f9128071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.389+0000 7f9127fff640 1 -- 172.21.15.133:0/1483866899 learned_addr learned my addr 172.21.15.133:0/1483866899 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:25:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.391+0000 7f91277fe640 1 -- 172.21.15.133:0/1483866899 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633828206 0 0) 0x7f9128158230 con 0x7f91280c53a0 2024-02-21T23:25:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.391+0000 7f91277fe640 1 -- 172.21.15.133:0/1483866899 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9110003520 con 0x7f91280c53a0 2024-02-21T23:25:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.391+0000 7f912d2bb640 1 -- 172.21.15.133:0/1483866899 >> v1:172.21.15.142:6789/0 conn(0x7f91280c4fc0 legacy=0x7f912806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.391+0000 7f912d2bb640 1 -- 172.21.15.133:0/1483866899 >> v1:172.21.15.133:6789/0 conn(0x7f91280c53a0 legacy=0x7f912815b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.392+0000 7f912d2bb640 1 -- 172.21.15.133:0/1483866899 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91281587f0 con 0x7f912815bec0 2024-02-21T23:25:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.396+0000 7f912d2bb640 1 --2- 172.21.15.133:0/1483866899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f912815bec0 0x7f9128071b60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9120009d90 tx=0x7f912002bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d55c812082d4603d server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.400+0000 7f91277fe640 1 -- 172.21.15.133:0/1483866899 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120039070 con 0x7f912815bec0 2024-02-21T23:25:58.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.400+0000 7f91277fe640 1 -- 172.21.15.133:0/1483866899 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912002cd80 con 0x7f912815bec0 2024-02-21T23:25:58.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.400+0000 7f91277fe640 1 -- 172.21.15.133:0/1483866899 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120034660 con 0x7f912815bec0 2024-02-21T23:25:58.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.400+0000 7f912ed1e640 1 -- 172.21.15.133:0/1483866899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f912815bec0 msgr2=0x7f9128071b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:58.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.400+0000 7f912ed1e640 1 --2- 172.21.15.133:0/1483866899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f912815bec0 0x7f9128071b60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9120009d90 tx=0x7f912002bfd0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.401+0000 7f912ed1e640 1 -- 172.21.15.133:0/1483866899 shutdown_connections 2024-02-21T23:25:58.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.401+0000 7f912ed1e640 1 --2- 172.21.15.133:0/1483866899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f912815bec0 0x7f9128071b60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.401+0000 7f912ed1e640 1 -- 172.21.15.133:0/1483866899 >> 172.21.15.133:0/1483866899 conn(0x7f912806aec0 msgr2=0x7f912806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:58.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.401+0000 7f912ed1e640 1 -- 172.21.15.133:0/1483866899 shutdown_connections 2024-02-21T23:25:58.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.401+0000 7f912ed1e640 1 -- 172.21.15.133:0/1483866899 wait complete. 2024-02-21T23:25:58.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.402+0000 7f912ed1e640 1 Processor -- start 2024-02-21T23:25:58.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.402+0000 7f912ed1e640 1 -- start start 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912ed1e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c4fc0 0x7f91281ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912ed1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91280c53a0 0x7f9128144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912ed1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f912815bec0 0x7f91281452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912ed1e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912806f8d0 con 0x7f91280c53a0 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912ed1e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9128157f20 con 0x7f91280c4fc0 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912ed1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912806fa50 con 0x7f912815bec0 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f9127fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91280c53a0 0x7f9128144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912caba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c4fc0 0x7f91281ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912d2bb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f912815bec0 0x7f91281452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912caba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c4fc0 0x7f91281ee0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42880/0 (socket says 172.21.15.133:42880) 2024-02-21T23:25:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.403+0000 7f912caba640 1 -- 172.21.15.133:0/2247607633 learned_addr learned my addr 172.21.15.133:0/2247607633 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.407+0000 7f9127fff640 1 -- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f912815bec0 msgr2=0x7f91281452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.407+0000 7f9127fff640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f912815bec0 0x7f91281452e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.407+0000 7f9127fff640 1 -- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c4fc0 msgr2=0x7f91281ee0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.407+0000 7f9127fff640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c4fc0 0x7f91281ee0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.407+0000 7f9127fff640 1 -- 172.21.15.133:0/2247607633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128149c50 con 0x7f91280c53a0 2024-02-21T23:25:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.409+0000 7f9127fff640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91280c53a0 0x7f9128144320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f91280c57b0 tx=0x7f911c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:58.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.413+0000 7f91257fa640 1 -- 172.21.15.133:0/2247607633 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c017070 con 0x7f91280c53a0 2024-02-21T23:25:58.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.413+0000 7f91257fa640 1 -- 172.21.15.133:0/2247607633 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c005ce0 con 0x7f91280c53a0 2024-02-21T23:25:58.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.413+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91281490d0 con 0x7f91280c53a0 2024-02-21T23:25:58.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.413+0000 7f91257fa640 1 -- 172.21.15.133:0/2247607633 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0072d0 con 0x7f91280c53a0 2024-02-21T23:25:58.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.413+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9128149690 con 0x7f91280c53a0 2024-02-21T23:25:58.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.414+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912806a130 con 0x7f91280c53a0 2024-02-21T23:25:58.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.433+0000 7f91257fa640 1 -- 172.21.15.133:0/2247607633 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f911c01a070 con 0x7f91280c53a0 2024-02-21T23:25:58.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.434+0000 7f91257fa640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f911406b500 0x7f911406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:58.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.434+0000 7f912caba640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f911406b500 0x7f911406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:58.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.435+0000 7f912caba640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f911406b500 0x7f911406d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9118001840 tx=0x7f9118009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.671+0000 7f91257fa640 1 -- 172.21.15.133:0/2247607633 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f911c0561f0 con 0x7f91280c53a0 2024-02-21T23:25:58.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.671+0000 7f91257fa640 1 -- 172.21.15.133:0/2247607633 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f911c058150 con 0x7f91280c53a0 2024-02-21T23:25:58.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.855+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9128071960 con 0x7f911406b500 2024-02-21T23:25:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.863+0000 7f91257fa640 1 -- 172.21.15.133:0/2247607633 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332723 (secure 0 0 0) 0x7f9128071960 con 0x7f911406b500 2024-02-21T23:25:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:58.854 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-02-21T23:25:58.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f911406b500 msgr2=0x7f911406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:58.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f911406b500 0x7f911406d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9118001840 tx=0x7f9118009040 comp rx=0 tx=0).stop 2024-02-21T23:25:58.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91280c53a0 msgr2=0x7f9128144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:58.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91280c53a0 0x7f9128144320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f91280c57b0 tx=0x7f911c00c890 comp rx=0 tx=0).stop 2024-02-21T23:25:58.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 shutdown_connections 2024-02-21T23:25:58.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f911406b500 0x7f911406d9c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f912815bec0 0x7f91281452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91280c53a0 0x7f9128144320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 --2- 172.21.15.133:0/2247607633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c4fc0 0x7f91281ee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:58.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 >> 172.21.15.133:0/2247607633 conn(0x7f912806aec0 msgr2=0x7f91280c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:58.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 shutdown_connections 2024-02-21T23:25:58.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:58.867+0000 7f912ed1e640 1 -- 172.21.15.133:0/2247607633 wait complete. 2024-02-21T23:25:58.864 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-02-21T23:25:58.693229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":328,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218540,"kb_used_data":2356,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756180,"statfs":{"total":773094113280,"available":772870328320,"internally_reserved":0,"allocated":2412544,"data_stored":2099994,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001688"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519897+0000","last_change":"2024-02-21T23:25:16.134390+0000","last_active":"2024-02-21T23:25:57.519897+0000","last_peered":"2024-02-21T23:25:57.519897+0000","last_clean":"2024-02-21T23:25:57.519897+0000","last_became_active":"2024-02-21T23:25:16.134112+0000","last_became_peered":"2024-02-21T23:25:16.134112+0000","last_unstale":"2024-02-21T23:25:57.519897+0000","last_undegraded":"2024-02-21T23:25:57.519897+0000","last_fullsized":"2024-02-21T23:25:57.519897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.520728+0000","last_change":"2024-02-21T23:25:17.135738+0000","last_active":"2024-02-21T23:25:57.520728+0000","last_peered":"2024-02-21T23:25:57.520728+0000","last_clean":"2024-02-21T23:25:57.520728+0000","last_became_active":"2024-02-21T23:25:17.135586+0000","last_became_peered":"2024-02-21T23:25:17.135586+0000","last_unstale":"2024-02-21T23:25:57.520728+0000","last_undegraded":"2024-02-21T23:25:57.520728+0000","last_fullsized":"2024-02-21T23:25:57.520728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518429+0000","last_change":"2024-02-21T23:25:22.248962+0000","last_active":"2024-02-21T23:25:57.518429+0000","last_peered":"2024-02-21T23:25:57.518429+0000","last_clean":"2024-02-21T23:25:57.518429+0000","last_became_active":"2024-02-21T23:25:22.248610+0000","last_became_peered":"2024-02-21T23:25:22.248610+0000","last_unstale":"2024-02-21T23:25:57.518429+0000","last_undegraded":"2024-02-21T23:25:57.518429+0000","last_fullsized":"2024-02-21T23:25:57.518429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:45.939036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.520893+0000","last_change":"2024-02-21T23:25:16.123635+0000","last_active":"2024-02-21T23:25:57.520893+0000","last_peered":"2024-02-21T23:25:57.520893+0000","last_clean":"2024-02-21T23:25:57.520893+0000","last_became_active":"2024-02-21T23:25:16.123374+0000","last_became_peered":"2024-02-21T23:25:16.123374+0000","last_unstale":"2024-02-21T23:25:57.520893+0000","last_undegraded":"2024-02-21T23:25:57.520893+0000","last_fullsized":"2024-02-21T23:25:57.520893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519645+0000","last_change":"2024-02-21T23:25:17.192858+0000","last_active":"2024-02-21T23:25:57.519645+0000","last_peered":"2024-02-21T23:25:57.519645+0000","last_clean":"2024-02-21T23:25:57.519645+0000","last_became_active":"2024-02-21T23:25:17.192626+0000","last_became_peered":"2024-02-21T23:25:17.192626+0000","last_unstale":"2024-02-21T23:25:57.519645+0000","last_undegraded":"2024-02-21T23:25:57.519645+0000","last_fullsized":"2024-02-21T23:25:57.519645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.583837+0000","last_change":"2024-02-21T23:25:57.583837+0000","last_active":"2024-02-21T23:25:57.583837+0000","last_peered":"2024-02-21T23:25:57.583837+0000","last_clean":"2024-02-21T23:25:57.583837+0000","last_became_active":"2024-02-21T23:25:57.583416+0000","last_became_peered":"2024-02-21T23:25:57.583416+0000","last_unstale":"2024-02-21T23:25:57.583837+0000","last_undegraded":"2024-02-21T23:25:57.583837+0000","last_fullsized":"2024-02-21T23:25:57.583837+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:25:24.780185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599356+0000","last_change":"2024-02-21T23:25:16.134805+0000","last_active":"2024-02-21T23:25:57.599356+0000","last_peered":"2024-02-21T23:25:57.599356+0000","last_clean":"2024-02-21T23:25:57.599356+0000","last_became_active":"2024-02-21T23:25:16.134533+0000","last_became_peered":"2024-02-21T23:25:16.134533+0000","last_unstale":"2024-02-21T23:25:57.599356+0000","last_undegraded":"2024-02-21T23:25:57.599356+0000","last_fullsized":"2024-02-21T23:25:57.599356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524121+0000","last_change":"2024-02-21T23:25:17.182131+0000","last_active":"2024-02-21T23:25:57.524121+0000","last_peered":"2024-02-21T23:25:57.524121+0000","last_clean":"2024-02-21T23:25:57.524121+0000","last_became_active":"2024-02-21T23:25:17.181886+0000","last_became_peered":"2024-02-21T23:25:17.181886+0000","last_unstale":"2024-02-21T23:25:57.524121+0000","last_undegraded":"2024-02-21T23:25:57.524121+0000","last_fullsized":"2024-02-21T23:25:57.524121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.515619+0000","last_change":"2024-02-21T23:25:21.683506+0000","last_active":"2024-02-21T23:25:57.515619+0000","last_peered":"2024-02-21T23:25:57.515619+0000","last_clean":"2024-02-21T23:25:57.515619+0000","last_became_active":"2024-02-21T23:25:21.683115+0000","last_became_peered":"2024-02-21T23:25:21.683115+0000","last_unstale":"2024-02-21T23:25:57.515619+0000","last_undegraded":"2024-02-21T23:25:57.515619+0000","last_fullsized":"2024-02-21T23:25:57.515619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:13.865145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.598992+0000","last_change":"2024-02-21T23:25:16.130851+0000","last_active":"2024-02-21T23:25:57.598992+0000","last_peered":"2024-02-21T23:25:57.598992+0000","last_clean":"2024-02-21T23:25:57.598992+0000","last_became_active":"2024-02-21T23:25:16.130604+0000","last_became_peered":"2024-02-21T23:25:16.130604+0000","last_unstale":"2024-02-21T23:25:57.598992+0000","last_undegraded":"2024-02-21T23:25:57.598992+0000","last_fullsized":"2024-02-21T23:25:57.598992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524365+0000","last_change":"2024-02-21T23:25:17.192642+0000","last_active":"2024-02-21T23:25:57.524365+0000","last_peered":"2024-02-21T23:25:57.524365+0000","last_clean":"2024-02-21T23:25:57.524365+0000","last_became_active":"2024-02-21T23:25:17.192394+0000","last_became_peered":"2024-02-21T23:25:17.192394+0000","last_unstale":"2024-02-21T23:25:57.524365+0000","last_undegraded":"2024-02-21T23:25:57.524365+0000","last_fullsized":"2024-02-21T23:25:57.524365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.515435+0000","last_change":"2024-02-21T23:25:21.679491+0000","last_active":"2024-02-21T23:25:57.515435+0000","last_peered":"2024-02-21T23:25:57.515435+0000","last_clean":"2024-02-21T23:25:57.515435+0000","last_became_active":"2024-02-21T23:25:21.679117+0000","last_became_peered":"2024-02-21T23:25:21.679117+0000","last_unstale":"2024-02-21T23:25:57.515435+0000","last_undegraded":"2024-02-21T23:25:57.515435+0000","last_fullsized":"2024-02-21T23:25:57.515435+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:52.680971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.581650+0000","last_change":"2024-02-21T23:25:17.161438+0000","last_active":"2024-02-21T23:25:57.581650+0000","last_peered":"2024-02-21T23:25:57.581650+0000","last_clean":"2024-02-21T23:25:57.581650+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:25:57.581650+0000","last_undegraded":"2024-02-21T23:25:57.581650+0000","last_fullsized":"2024-02-21T23:25:57.581650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.598992+0000","last_change":"2024-02-21T23:25:16.136913+0000","last_active":"2024-02-21T23:25:57.598992+0000","last_peered":"2024-02-21T23:25:57.598992+0000","last_clean":"2024-02-21T23:25:57.598992+0000","last_became_active":"2024-02-21T23:25:16.136628+0000","last_became_peered":"2024-02-21T23:25:16.136628+0000","last_unstale":"2024-02-21T23:25:57.598992+0000","last_undegraded":"2024-02-21T23:25:57.598992+0000","last_fullsized":"2024-02-21T23:25:57.598992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.526324+0000","last_change":"2024-02-21T23:25:57.526324+0000","last_active":"2024-02-21T23:25:57.526324+0000","last_peered":"2024-02-21T23:25:57.526324+0000","last_clean":"2024-02-21T23:25:57.526324+0000","last_became_active":"2024-02-21T23:25:57.525854+0000","last_became_peered":"2024-02-21T23:25:57.525854+0000","last_unstale":"2024-02-21T23:25:57.526324+0000","last_undegraded":"2024-02-21T23:25:57.526324+0000","last_fullsized":"2024-02-21T23:25:57.526324+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:12:32.356243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.520192+0000","last_change":"2024-02-21T23:25:17.137829+0000","last_active":"2024-02-21T23:25:57.520192+0000","last_peered":"2024-02-21T23:25:57.520192+0000","last_clean":"2024-02-21T23:25:57.520192+0000","last_became_active":"2024-02-21T23:25:17.137557+0000","last_became_peered":"2024-02-21T23:25:17.137557+0000","last_unstale":"2024-02-21T23:25:57.520192+0000","last_undegraded":"2024-02-21T23:25:57.520192+0000","last_fullsized":"2024-02-21T23:25:57.520192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599729+0000","last_change":"2024-02-21T23:25:16.146181+0000","last_active":"2024-02-21T23:25:57.599729+0000","last_peered":"2024-02-21T23:25:57.599729+0000","last_clean":"2024-02-21T23:25:57.599729+0000","last_became_active":"2024-02-21T23:25:16.145932+0000","last_became_peered":"2024-02-21T23:25:16.145932+0000","last_unstale":"2024-02-21T23:25:57.599729+0000","last_undegraded":"2024-02-21T23:25:57.599729+0000","last_fullsized":"2024-02-21T23:25:57.599729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.515636+0000","last_change":"2024-02-21T23:25:21.686044+0000","last_active":"2024-02-21T23:25:57.515636+0000","last_peered":"2024-02-21T23:25:57.515636+0000","last_clean":"2024-02-21T23:25:57.515636+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:25:57.515636+0000","last_undegraded":"2024-02-21T23:25:57.515636+0000","last_fullsized":"2024-02-21T23:25:57.515636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:39.740454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519894+0000","last_change":"2024-02-21T23:25:17.190236+0000","last_active":"2024-02-21T23:25:57.519894+0000","last_peered":"2024-02-21T23:25:57.519894+0000","last_clean":"2024-02-21T23:25:57.519894+0000","last_became_active":"2024-02-21T23:25:17.189957+0000","last_became_peered":"2024-02-21T23:25:17.189957+0000","last_unstale":"2024-02-21T23:25:57.519894+0000","last_undegraded":"2024-02-21T23:25:57.519894+0000","last_fullsized":"2024-02-21T23:25:57.519894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599256+0000","last_change":"2024-02-21T23:25:16.148108+0000","last_active":"2024-02-21T23:25:57.599256+0000","last_peered":"2024-02-21T23:25:57.599256+0000","last_clean":"2024-02-21T23:25:57.599256+0000","last_became_active":"2024-02-21T23:25:16.147930+0000","last_became_peered":"2024-02-21T23:25:16.147930+0000","last_unstale":"2024-02-21T23:25:57.599256+0000","last_undegraded":"2024-02-21T23:25:57.599256+0000","last_fullsized":"2024-02-21T23:25:57.599256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600020+0000","last_change":"2024-02-21T23:25:22.248150+0000","last_active":"2024-02-21T23:25:57.600020+0000","last_peered":"2024-02-21T23:25:57.600020+0000","last_clean":"2024-02-21T23:25:57.600020+0000","last_became_active":"2024-02-21T23:25:22.247769+0000","last_became_peered":"2024-02-21T23:25:22.247769+0000","last_unstale":"2024-02-21T23:25:57.600020+0000","last_undegraded":"2024-02-21T23:25:57.600020+0000","last_fullsized":"2024-02-21T23:25:57.600020+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.515361+0000","last_change":"2024-02-21T23:25:17.189758+0000","last_active":"2024-02-21T23:25:57.515361+0000","last_peered":"2024-02-21T23:25:57.515361+0000","last_clean":"2024-02-21T23:25:57.515361+0000","last_became_active":"2024-02-21T23:25:17.189492+0000","last_became_peered":"2024-02-21T23:25:17.189492+0000","last_unstale":"2024-02-21T23:25:57.515361+0000","last_undegraded":"2024-02-21T23:25:57.515361+0000","last_fullsized":"2024-02-21T23:25:57.515361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:42.526598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.598624+0000","last_change":"2024-02-21T23:25:16.131611+0000","last_active":"2024-02-21T23:25:57.598624+0000","last_peered":"2024-02-21T23:25:57.598624+0000","last_clean":"2024-02-21T23:25:57.598624+0000","last_became_active":"2024-02-21T23:25:16.131158+0000","last_became_peered":"2024-02-21T23:25:16.131158+0000","last_unstale":"2024-02-21T23:25:57.598624+0000","last_undegraded":"2024-02-21T23:25:57.598624+0000","last_fullsized":"2024-02-21T23:25:57.598624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.516288+0000","last_change":"2024-02-21T23:25:21.683821+0000","last_active":"2024-02-21T23:25:57.516288+0000","last_peered":"2024-02-21T23:25:57.516288+0000","last_clean":"2024-02-21T23:25:57.516288+0000","last_became_active":"2024-02-21T23:25:21.683466+0000","last_became_peered":"2024-02-21T23:25:21.683466+0000","last_unstale":"2024-02-21T23:25:57.516288+0000","last_undegraded":"2024-02-21T23:25:57.516288+0000","last_fullsized":"2024-02-21T23:25:57.516288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518675+0000","last_change":"2024-02-21T23:25:17.191394+0000","last_active":"2024-02-21T23:25:57.518675+0000","last_peered":"2024-02-21T23:25:57.518675+0000","last_clean":"2024-02-21T23:25:57.518675+0000","last_became_active":"2024-02-21T23:25:17.191106+0000","last_became_peered":"2024-02-21T23:25:17.191106+0000","last_unstale":"2024-02-21T23:25:57.518675+0000","last_undegraded":"2024-02-21T23:25:57.518675+0000","last_fullsized":"2024-02-21T23:25:57.518675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.582125+0000","last_change":"2024-02-21T23:25:16.134862+0000","last_active":"2024-02-21T23:25:57.582125+0000","last_peered":"2024-02-21T23:25:57.582125+0000","last_clean":"2024-02-21T23:25:57.582125+0000","last_became_active":"2024-02-21T23:25:16.134602+0000","last_became_peered":"2024-02-21T23:25:16.134602+0000","last_unstale":"2024-02-21T23:25:57.582125+0000","last_undegraded":"2024-02-21T23:25:57.582125+0000","last_fullsized":"2024-02-21T23:25:57.582125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.515249+0000","last_change":"2024-02-21T23:25:22.249606+0000","last_active":"2024-02-21T23:25:57.515249+0000","last_peered":"2024-02-21T23:25:57.515249+0000","last_clean":"2024-02-21T23:25:57.515249+0000","last_became_active":"2024-02-21T23:25:22.249214+0000","last_became_peered":"2024-02-21T23:25:22.249214+0000","last_unstale":"2024-02-21T23:25:57.515249+0000","last_undegraded":"2024-02-21T23:25:57.515249+0000","last_fullsized":"2024-02-21T23:25:57.515249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:36.929076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.521170+0000","last_change":"2024-02-21T23:25:17.183290+0000","last_active":"2024-02-21T23:25:57.521170+0000","last_peered":"2024-02-21T23:25:57.521170+0000","last_clean":"2024-02-21T23:25:57.521170+0000","last_became_active":"2024-02-21T23:25:17.183047+0000","last_became_peered":"2024-02-21T23:25:17.183047+0000","last_unstale":"2024-02-21T23:25:57.521170+0000","last_undegraded":"2024-02-21T23:25:57.521170+0000","last_fullsized":"2024-02-21T23:25:57.521170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600236+0000","last_change":"2024-02-21T23:25:16.146202+0000","last_active":"2024-02-21T23:25:57.600236+0000","last_peered":"2024-02-21T23:25:57.600236+0000","last_clean":"2024-02-21T23:25:57.600236+0000","last_became_active":"2024-02-21T23:25:16.145911+0000","last_became_peered":"2024-02-21T23:25:16.145911+0000","last_unstale":"2024-02-21T23:25:57.600236+0000","last_undegraded":"2024-02-21T23:25:57.600236+0000","last_fullsized":"2024-02-21T23:25:57.600236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519467+0000","last_change":"2024-02-21T23:25:22.243297+0000","last_active":"2024-02-21T23:25:57.519467+0000","last_peered":"2024-02-21T23:25:57.519467+0000","last_clean":"2024-02-21T23:25:57.519467+0000","last_became_active":"2024-02-21T23:25:22.242922+0000","last_became_peered":"2024-02-21T23:25:22.242922+0000","last_unstale":"2024-02-21T23:25:57.519467+0000","last_undegraded":"2024-02-21T23:25:57.519467+0000","last_fullsized":"2024-02-21T23:25:57.519467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:52:51.773574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600116+0000","last_change":"2024-02-21T23:25:17.190577+0000","last_active":"2024-02-21T23:25:57.600116+0000","last_peered":"2024-02-21T23:25:57.600116+0000","last_clean":"2024-02-21T23:25:57.600116+0000","last_became_active":"2024-02-21T23:25:17.190332+0000","last_became_peered":"2024-02-21T23:25:17.190332+0000","last_unstale":"2024-02-21T23:25:57.600116+0000","last_undegraded":"2024-02-21T23:25:57.600116+0000","last_fullsized":"2024-02-21T23:25:57.600116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599269+0000","last_change":"2024-02-21T23:25:16.134076+0000","last_active":"2024-02-21T23:25:57.599269+0000","last_peered":"2024-02-21T23:25:57.599269+0000","last_clean":"2024-02-21T23:25:57.599269+0000","last_became_active":"2024-02-21T23:25:16.133816+0000","last_became_peered":"2024-02-21T23:25:16.133816+0000","last_unstale":"2024-02-21T23:25:57.599269+0000","last_undegraded":"2024-02-21T23:25:57.599269+0000","last_fullsized":"2024-02-21T23:25:57.599269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519026+0000","last_change":"2024-02-21T23:25:22.243360+0000","last_active":"2024-02-21T23:25:57.519026+0000","last_peered":"2024-02-21T23:25:57.519026+0000","last_clean":"2024-02-21T23:25:57.519026+0000","last_became_active":"2024-02-21T23:25:22.242958+0000","last_became_peered":"2024-02-21T23:25:22.242958+0000","last_unstale":"2024-02-21T23:25:57.519026+0000","last_undegraded":"2024-02-21T23:25:57.519026+0000","last_fullsized":"2024-02-21T23:25:57.519026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524396+0000","last_change":"2024-02-21T23:25:17.192402+0000","last_active":"2024-02-21T23:25:57.524396+0000","last_peered":"2024-02-21T23:25:57.524396+0000","last_clean":"2024-02-21T23:25:57.524396+0000","last_became_active":"2024-02-21T23:25:17.192138+0000","last_became_peered":"2024-02-21T23:25:17.192138+0000","last_unstale":"2024-02-21T23:25:57.524396+0000","last_undegraded":"2024-02-21T23:25:57.524396+0000","last_fullsized":"2024-02-21T23:25:57.524396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600212+0000","last_change":"2024-02-21T23:25:16.145521+0000","last_active":"2024-02-21T23:25:57.600212+0000","last_peered":"2024-02-21T23:25:57.600212+0000","last_clean":"2024-02-21T23:25:57.600212+0000","last_became_active":"2024-02-21T23:25:16.145255+0000","last_became_peered":"2024-02-21T23:25:16.145255+0000","last_unstale":"2024-02-21T23:25:57.600212+0000","last_undegraded":"2024-02-21T23:25:57.600212+0000","last_fullsized":"2024-02-21T23:25:57.600212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599329+0000","last_change":"2024-02-21T23:25:21.683849+0000","last_active":"2024-02-21T23:25:57.599329+0000","last_peered":"2024-02-21T23:25:57.599329+0000","last_clean":"2024-02-21T23:25:57.599329+0000","last_became_active":"2024-02-21T23:25:21.683379+0000","last_became_peered":"2024-02-21T23:25:21.683379+0000","last_unstale":"2024-02-21T23:25:57.599329+0000","last_undegraded":"2024-02-21T23:25:57.599329+0000","last_fullsized":"2024-02-21T23:25:57.599329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.515900+0000","last_change":"2024-02-21T23:25:17.166232+0000","last_active":"2024-02-21T23:25:57.515900+0000","last_peered":"2024-02-21T23:25:57.515900+0000","last_clean":"2024-02-21T23:25:57.515900+0000","last_became_active":"2024-02-21T23:25:17.165999+0000","last_became_peered":"2024-02-21T23:25:17.165999+0000","last_unstale":"2024-02-21T23:25:57.515900+0000","last_undegraded":"2024-02-21T23:25:57.515900+0000","last_fullsized":"2024-02-21T23:25:57.515900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:45.125986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523957+0000","last_change":"2024-02-21T23:25:16.147082+0000","last_active":"2024-02-21T23:25:57.523957+0000","last_peered":"2024-02-21T23:25:57.523957+0000","last_clean":"2024-02-21T23:25:57.523957+0000","last_became_active":"2024-02-21T23:25:16.146797+0000","last_became_peered":"2024-02-21T23:25:16.146797+0000","last_unstale":"2024-02-21T23:25:57.523957+0000","last_undegraded":"2024-02-21T23:25:57.523957+0000","last_fullsized":"2024-02-21T23:25:57.523957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.584307+0000","last_change":"2024-02-21T23:25:57.583768+0000","last_active":"2024-02-21T23:25:57.584307+0000","last_peered":"2024-02-21T23:25:57.584307+0000","last_clean":"2024-02-21T23:25:57.584307+0000","last_became_active":"2024-02-21T23:25:57.583297+0000","last_became_peered":"2024-02-21T23:25:57.583297+0000","last_unstale":"2024-02-21T23:25:57.584307+0000","last_undegraded":"2024-02-21T23:25:57.584307+0000","last_fullsized":"2024-02-21T23:25:57.584307+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:19:14.798657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,6],"acting":[0,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599321+0000","last_change":"2024-02-21T23:25:17.190425+0000","last_active":"2024-02-21T23:25:57.599321+0000","last_peered":"2024-02-21T23:25:57.599321+0000","last_clean":"2024-02-21T23:25:57.599321+0000","last_became_active":"2024-02-21T23:25:17.190172+0000","last_became_peered":"2024-02-21T23:25:17.190172+0000","last_unstale":"2024-02-21T23:25:57.599321+0000","last_undegraded":"2024-02-21T23:25:57.599321+0000","last_fullsized":"2024-02-21T23:25:57.599321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524300+0000","last_change":"2024-02-21T23:25:16.108263+0000","last_active":"2024-02-21T23:25:57.524300+0000","last_peered":"2024-02-21T23:25:57.524300+0000","last_clean":"2024-02-21T23:25:57.524300+0000","last_became_active":"2024-02-21T23:25:16.107822+0000","last_became_peered":"2024-02-21T23:25:16.107822+0000","last_unstale":"2024-02-21T23:25:57.524300+0000","last_undegraded":"2024-02-21T23:25:57.524300+0000","last_fullsized":"2024-02-21T23:25:57.524300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600106+0000","last_change":"2024-02-21T23:25:21.686034+0000","last_active":"2024-02-21T23:25:57.600106+0000","last_peered":"2024-02-21T23:25:57.600106+0000","last_clean":"2024-02-21T23:25:57.600106+0000","last_became_active":"2024-02-21T23:25:21.685546+0000","last_became_peered":"2024-02-21T23:25:21.685546+0000","last_unstale":"2024-02-21T23:25:57.600106+0000","last_undegraded":"2024-02-21T23:25:57.600106+0000","last_fullsized":"2024-02-21T23:25:57.600106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518308+0000","last_change":"2024-02-21T23:25:17.161738+0000","last_active":"2024-02-21T23:25:57.518308+0000","last_peered":"2024-02-21T23:25:57.518308+0000","last_clean":"2024-02-21T23:25:57.518308+0000","last_became_active":"2024-02-21T23:25:17.161459+0000","last_became_peered":"2024-02-21T23:25:17.161459+0000","last_unstale":"2024-02-21T23:25:57.518308+0000","last_undegraded":"2024-02-21T23:25:57.518308+0000","last_fullsized":"2024-02-21T23:25:57.518308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:31:58.434561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.598621+0000","last_change":"2024-02-21T23:25:16.147162+0000","last_active":"2024-02-21T23:25:57.598621+0000","last_peered":"2024-02-21T23:25:57.598621+0000","last_clean":"2024-02-21T23:25:57.598621+0000","last_became_active":"2024-02-21T23:25:16.146850+0000","last_became_peered":"2024-02-21T23:25:16.146850+0000","last_unstale":"2024-02-21T23:25:57.598621+0000","last_undegraded":"2024-02-21T23:25:57.598621+0000","last_fullsized":"2024-02-21T23:25:57.598621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518530+0000","last_change":"2024-02-21T23:25:22.248109+0000","last_active":"2024-02-21T23:25:57.518530+0000","last_peered":"2024-02-21T23:25:57.518530+0000","last_clean":"2024-02-21T23:25:57.518530+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:25:57.518530+0000","last_undegraded":"2024-02-21T23:25:57.518530+0000","last_fullsized":"2024-02-21T23:25:57.518530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600968+0000","last_change":"2024-02-21T23:25:57.600968+0000","last_active":"2024-02-21T23:25:57.600968+0000","last_peered":"2024-02-21T23:25:57.600968+0000","last_clean":"2024-02-21T23:25:57.600968+0000","last_became_active":"2024-02-21T23:25:57.600644+0000","last_became_peered":"2024-02-21T23:25:57.600644+0000","last_unstale":"2024-02-21T23:25:57.600968+0000","last_undegraded":"2024-02-21T23:25:57.600968+0000","last_fullsized":"2024-02-21T23:25:57.600968+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:13:09.640854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524016+0000","last_change":"2024-02-21T23:25:16.127688+0000","last_active":"2024-02-21T23:25:57.524016+0000","last_peered":"2024-02-21T23:25:57.524016+0000","last_clean":"2024-02-21T23:25:57.524016+0000","last_became_active":"2024-02-21T23:25:16.127388+0000","last_became_peered":"2024-02-21T23:25:16.127388+0000","last_unstale":"2024-02-21T23:25:57.524016+0000","last_undegraded":"2024-02-21T23:25:57.524016+0000","last_fullsized":"2024-02-21T23:25:57.524016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.598739+0000","last_change":"2024-02-21T23:25:05.612817+0000","last_active":"2024-02-21T23:25:57.598739+0000","last_peered":"2024-02-21T23:25:57.598739+0000","last_clean":"2024-02-21T23:25:57.598739+0000","last_became_active":"2024-02-21T23:25:05.612483+0000","last_became_peered":"2024-02-21T23:25:05.612483+0000","last_unstale":"2024-02-21T23:25:57.598739+0000","last_undegraded":"2024-02-21T23:25:57.598739+0000","last_fullsized":"2024-02-21T23:25:57.598739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_clean_scrub_stamp":"2024-02-21T23:25:03.736732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:23:33.073040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.581862+0000","last_change":"2024-02-21T23:25:22.248174+0000","last_active":"2024-02-21T23:25:57.581862+0000","last_peered":"2024-02-21T23:25:57.581862+0000","last_clean":"2024-02-21T23:25:57.581862+0000","last_became_active":"2024-02-21T23:25:22.247834+0000","last_became_peered":"2024-02-21T23:25:22.247834+0000","last_unstale":"2024-02-21T23:25:57.581862+0000","last_undegraded":"2024-02-21T23:25:57.581862+0000","last_fullsized":"2024-02-21T23:25:57.581862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:13.953358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599066+0000","last_change":"2024-02-21T23:25:17.192575+0000","last_active":"2024-02-21T23:25:57.599066+0000","last_peered":"2024-02-21T23:25:57.599066+0000","last_clean":"2024-02-21T23:25:57.599066+0000","last_became_active":"2024-02-21T23:25:17.192293+0000","last_became_peered":"2024-02-21T23:25:17.192293+0000","last_unstale":"2024-02-21T23:25:57.599066+0000","last_undegraded":"2024-02-21T23:25:57.599066+0000","last_fullsized":"2024-02-21T23:25:57.599066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523569+0000","last_change":"2024-02-21T23:25:16.128369+0000","last_active":"2024-02-21T23:25:57.523569+0000","last_peered":"2024-02-21T23:25:57.523569+0000","last_clean":"2024-02-21T23:25:57.523569+0000","last_became_active":"2024-02-21T23:25:16.128067+0000","last_became_peered":"2024-02-21T23:25:16.128067+0000","last_unstale":"2024-02-21T23:25:57.523569+0000","last_undegraded":"2024-02-21T23:25:57.523569+0000","last_fullsized":"2024-02-21T23:25:57.523569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.581772+0000","last_change":"2024-02-21T23:25:22.243282+0000","last_active":"2024-02-21T23:25:57.581772+0000","last_peered":"2024-02-21T23:25:57.581772+0000","last_clean":"2024-02-21T23:25:57.581772+0000","last_became_active":"2024-02-21T23:25:22.242916+0000","last_became_peered":"2024-02-21T23:25:22.242916+0000","last_unstale":"2024-02-21T23:25:57.581772+0000","last_undegraded":"2024-02-21T23:25:57.581772+0000","last_fullsized":"2024-02-21T23:25:57.581772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524125+0000","last_change":"2024-02-21T23:25:17.185418+0000","last_active":"2024-02-21T23:25:57.524125+0000","last_peered":"2024-02-21T23:25:57.524125+0000","last_clean":"2024-02-21T23:25:57.524125+0000","last_became_active":"2024-02-21T23:25:17.185119+0000","last_became_peered":"2024-02-21T23:25:17.185119+0000","last_unstale":"2024-02-21T23:25:57.524125+0000","last_undegraded":"2024-02-21T23:25:57.524125+0000","last_fullsized":"2024-02-21T23:25:57.524125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600520+0000","last_change":"2024-02-21T23:25:16.151088+0000","last_active":"2024-02-21T23:25:57.600520+0000","last_peered":"2024-02-21T23:25:57.600520+0000","last_clean":"2024-02-21T23:25:57.600520+0000","last_became_active":"2024-02-21T23:25:16.150819+0000","last_became_peered":"2024-02-21T23:25:16.150819+0000","last_unstale":"2024-02-21T23:25:57.600520+0000","last_undegraded":"2024-02-21T23:25:57.600520+0000","last_fullsized":"2024-02-21T23:25:57.600520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519205+0000","last_change":"2024-02-21T23:25:21.684592+0000","last_active":"2024-02-21T23:25:57.519205+0000","last_peered":"2024-02-21T23:25:57.519205+0000","last_clean":"2024-02-21T23:25:57.519205+0000","last_became_active":"2024-02-21T23:25:21.684192+0000","last_became_peered":"2024-02-21T23:25:21.684192+0000","last_unstale":"2024-02-21T23:25:57.519205+0000","last_undegraded":"2024-02-21T23:25:57.519205+0000","last_fullsized":"2024-02-21T23:25:57.519205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599392+0000","last_change":"2024-02-21T23:25:17.184268+0000","last_active":"2024-02-21T23:25:57.599392+0000","last_peered":"2024-02-21T23:25:57.599392+0000","last_clean":"2024-02-21T23:25:57.599392+0000","last_became_active":"2024-02-21T23:25:17.184027+0000","last_became_peered":"2024-02-21T23:25:17.184027+0000","last_unstale":"2024-02-21T23:25:57.599392+0000","last_undegraded":"2024-02-21T23:25:57.599392+0000","last_fullsized":"2024-02-21T23:25:57.599392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523246+0000","last_change":"2024-02-21T23:25:16.108099+0000","last_active":"2024-02-21T23:25:57.523246+0000","last_peered":"2024-02-21T23:25:57.523246+0000","last_clean":"2024-02-21T23:25:57.523246+0000","last_became_active":"2024-02-21T23:25:16.107815+0000","last_became_peered":"2024-02-21T23:25:16.107815+0000","last_unstale":"2024-02-21T23:25:57.523246+0000","last_undegraded":"2024-02-21T23:25:57.523246+0000","last_fullsized":"2024-02-21T23:25:57.523246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523256+0000","last_change":"2024-02-21T23:25:22.248926+0000","last_active":"2024-02-21T23:25:57.523256+0000","last_peered":"2024-02-21T23:25:57.523256+0000","last_clean":"2024-02-21T23:25:57.523256+0000","last_became_active":"2024-02-21T23:25:22.248558+0000","last_became_peered":"2024-02-21T23:25:22.248558+0000","last_unstale":"2024-02-21T23:25:57.523256+0000","last_undegraded":"2024-02-21T23:25:57.523256+0000","last_fullsized":"2024-02-21T23:25:57.523256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599221+0000","last_change":"2024-02-21T23:25:17.183254+0000","last_active":"2024-02-21T23:25:57.599221+0000","last_peered":"2024-02-21T23:25:57.599221+0000","last_clean":"2024-02-21T23:25:57.599221+0000","last_became_active":"2024-02-21T23:25:17.183020+0000","last_became_peered":"2024-02-21T23:25:17.183020+0000","last_unstale":"2024-02-21T23:25:57.599221+0000","last_undegraded":"2024-02-21T23:25:57.599221+0000","last_fullsized":"2024-02-21T23:25:57.599221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.516769+0000","last_change":"2024-02-21T23:25:16.129131+0000","last_active":"2024-02-21T23:25:57.516769+0000","last_peered":"2024-02-21T23:25:57.516769+0000","last_clean":"2024-02-21T23:25:57.516769+0000","last_became_active":"2024-02-21T23:25:16.128853+0000","last_became_peered":"2024-02-21T23:25:16.128853+0000","last_unstale":"2024-02-21T23:25:57.516769+0000","last_undegraded":"2024-02-21T23:25:57.516769+0000","last_fullsized":"2024-02-21T23:25:57.516769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:29.302181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.515020+0000","last_change":"2024-02-21T23:25:21.684342+0000","last_active":"2024-02-21T23:25:57.515020+0000","last_peered":"2024-02-21T23:25:57.515020+0000","last_clean":"2024-02-21T23:25:57.515020+0000","last_became_active":"2024-02-21T23:25:21.683991+0000","last_became_peered":"2024-02-21T23:25:21.683991+0000","last_unstale":"2024-02-21T23:25:57.515020+0000","last_undegraded":"2024-02-21T23:25:57.515020+0000","last_fullsized":"2024-02-21T23:25:57.515020+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:21:00.312314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.598839+0000","last_change":"2024-02-21T23:25:17.183442+0000","last_active":"2024-02-21T23:25:57.598839+0000","last_peered":"2024-02-21T23:25:57.598839+0000","last_clean":"2024-02-21T23:25:57.598839+0000","last_became_active":"2024-02-21T23:25:17.183057+0000","last_became_peered":"2024-02-21T23:25:17.183057+0000","last_unstale":"2024-02-21T23:25:57.598839+0000","last_undegraded":"2024-02-21T23:25:57.598839+0000","last_fullsized":"2024-02-21T23:25:57.598839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524024+0000","last_change":"2024-02-21T23:25:16.145801+0000","last_active":"2024-02-21T23:25:57.524024+0000","last_peered":"2024-02-21T23:25:57.524024+0000","last_clean":"2024-02-21T23:25:57.524024+0000","last_became_active":"2024-02-21T23:25:16.145523+0000","last_became_peered":"2024-02-21T23:25:16.145523+0000","last_unstale":"2024-02-21T23:25:57.524024+0000","last_undegraded":"2024-02-21T23:25:57.524024+0000","last_fullsized":"2024-02-21T23:25:57.524024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518435+0000","last_change":"2024-02-21T23:25:21.679191+0000","last_active":"2024-02-21T23:25:57.518435+0000","last_peered":"2024-02-21T23:25:57.518435+0000","last_clean":"2024-02-21T23:25:57.518435+0000","last_became_active":"2024-02-21T23:25:21.678715+0000","last_became_peered":"2024-02-21T23:25:21.678715+0000","last_unstale":"2024-02-21T23:25:57.518435+0000","last_undegraded":"2024-02-21T23:25:57.518435+0000","last_fullsized":"2024-02-21T23:25:57.518435+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.581996+0000","last_change":"2024-02-21T23:25:17.138401+0000","last_active":"2024-02-21T23:25:57.581996+0000","last_peered":"2024-02-21T23:25:57.581996+0000","last_clean":"2024-02-21T23:25:57.581996+0000","last_became_active":"2024-02-21T23:25:17.138136+0000","last_became_peered":"2024-02-21T23:25:17.138136+0000","last_unstale":"2024-02-21T23:25:57.581996+0000","last_undegraded":"2024-02-21T23:25:57.581996+0000","last_fullsized":"2024-02-21T23:25:57.581996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.581817+0000","last_change":"2024-02-21T23:25:16.130209+0000","last_active":"2024-02-21T23:25:57.581817+0000","last_peered":"2024-02-21T23:25:57.581817+0000","last_clean":"2024-02-21T23:25:57.581817+0000","last_became_active":"2024-02-21T23:25:16.129974+0000","last_became_peered":"2024-02-21T23:25:16.129974+0000","last_unstale":"2024-02-21T23:25:57.581817+0000","last_undegraded":"2024-02-21T23:25:57.581817+0000","last_fullsized":"2024-02-21T23:25:57.581817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.521081+0000","last_change":"2024-02-21T23:25:22.243303+0000","last_active":"2024-02-21T23:25:57.521081+0000","last_peered":"2024-02-21T23:25:57.521081+0000","last_clean":"2024-02-21T23:25:57.521081+0000","last_became_active":"2024-02-21T23:25:22.242960+0000","last_became_peered":"2024-02-21T23:25:22.242960+0000","last_unstale":"2024-02-21T23:25:57.521081+0000","last_undegraded":"2024-02-21T23:25:57.521081+0000","last_fullsized":"2024-02-21T23:25:57.521081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:54:16.386354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.598892+0000","last_change":"2024-02-21T23:25:17.184233+0000","last_active":"2024-02-21T23:25:57.598892+0000","last_peered":"2024-02-21T23:25:57.598892+0000","last_clean":"2024-02-21T23:25:57.598892+0000","last_became_active":"2024-02-21T23:25:17.183980+0000","last_became_peered":"2024-02-21T23:25:17.183980+0000","last_unstale":"2024-02-21T23:25:57.598892+0000","last_undegraded":"2024-02-21T23:25:57.598892+0000","last_fullsized":"2024-02-21T23:25:57.598892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523850+0000","last_change":"2024-02-21T23:25:16.131147+0000","last_active":"2024-02-21T23:25:57.523850+0000","last_peered":"2024-02-21T23:25:57.523850+0000","last_clean":"2024-02-21T23:25:57.523850+0000","last_became_active":"2024-02-21T23:25:16.130553+0000","last_became_peered":"2024-02-21T23:25:16.130553+0000","last_unstale":"2024-02-21T23:25:57.523850+0000","last_undegraded":"2024-02-21T23:25:57.523850+0000","last_fullsized":"2024-02-21T23:25:57.523850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.520535+0000","last_change":"2024-02-21T23:25:22.243261+0000","last_active":"2024-02-21T23:25:57.520535+0000","last_peered":"2024-02-21T23:25:57.520535+0000","last_clean":"2024-02-21T23:25:57.520535+0000","last_became_active":"2024-02-21T23:25:22.242900+0000","last_became_peered":"2024-02-21T23:25:22.242900+0000","last_unstale":"2024-02-21T23:25:57.520535+0000","last_undegraded":"2024-02-21T23:25:57.520535+0000","last_fullsized":"2024-02-21T23:25:57.520535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:34:44.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518806+0000","last_change":"2024-02-21T23:25:17.183303+0000","last_active":"2024-02-21T23:25:57.518806+0000","last_peered":"2024-02-21T23:25:57.518806+0000","last_clean":"2024-02-21T23:25:57.518806+0000","last_became_active":"2024-02-21T23:25:17.183042+0000","last_became_peered":"2024-02-21T23:25:17.183042+0000","last_unstale":"2024-02-21T23:25:57.518806+0000","last_undegraded":"2024-02-21T23:25:57.518806+0000","last_fullsized":"2024-02-21T23:25:57.518806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519475+0000","last_change":"2024-02-21T23:25:16.127609+0000","last_active":"2024-02-21T23:25:57.519475+0000","last_peered":"2024-02-21T23:25:57.519475+0000","last_clean":"2024-02-21T23:25:57.519475+0000","last_became_active":"2024-02-21T23:25:16.127375+0000","last_became_peered":"2024-02-21T23:25:16.127375+0000","last_unstale":"2024-02-21T23:25:57.519475+0000","last_undegraded":"2024-02-21T23:25:57.519475+0000","last_fullsized":"2024-02-21T23:25:57.519475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524241+0000","last_change":"2024-02-21T23:25:22.248329+0000","last_active":"2024-02-21T23:25:57.524241+0000","last_peered":"2024-02-21T23:25:57.524241+0000","last_clean":"2024-02-21T23:25:57.524241+0000","last_became_active":"2024-02-21T23:25:22.247889+0000","last_became_peered":"2024-02-21T23:25:22.247889+0000","last_unstale":"2024-02-21T23:25:57.524241+0000","last_undegraded":"2024-02-21T23:25:57.524241+0000","last_fullsized":"2024-02-21T23:25:57.524241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.530722+0000","last_change":"2024-02-21T23:25:57.527196+0000","last_active":"2024-02-21T23:25:57.530722+0000","last_peered":"2024-02-21T23:25:57.530722+0000","last_clean":"2024-02-21T23:25:57.530722+0000","last_became_active":"2024-02-21T23:25:57.526874+0000","last_became_peered":"2024-02-21T23:25:57.526874+0000","last_unstale":"2024-02-21T23:25:57.530722+0000","last_undegraded":"2024-02-21T23:25:57.530722+0000","last_fullsized":"2024-02-21T23:25:57.530722+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.582276+0000","last_change":"2024-02-21T23:25:16.116460+0000","last_active":"2024-02-21T23:25:57.582276+0000","last_peered":"2024-02-21T23:25:57.582276+0000","last_clean":"2024-02-21T23:25:57.582276+0000","last_became_active":"2024-02-21T23:25:16.116162+0000","last_became_peered":"2024-02-21T23:25:16.116162+0000","last_unstale":"2024-02-21T23:25:57.582276+0000","last_undegraded":"2024-02-21T23:25:57.582276+0000","last_fullsized":"2024-02-21T23:25:57.582276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599529+0000","last_change":"2024-02-21T23:25:21.683199+0000","last_active":"2024-02-21T23:25:57.599529+0000","last_peered":"2024-02-21T23:25:57.599529+0000","last_clean":"2024-02-21T23:25:57.599529+0000","last_became_active":"2024-02-21T23:25:21.682425+0000","last_became_peered":"2024-02-21T23:25:21.682425+0000","last_unstale":"2024-02-21T23:25:57.599529+0000","last_undegraded":"2024-02-21T23:25:57.599529+0000","last_fullsized":"2024-02-21T23:25:57.599529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.516216+0000","last_change":"2024-02-21T23:25:17.184837+0000","last_active":"2024-02-21T23:25:57.516216+0000","last_peered":"2024-02-21T23:25:57.516216+0000","last_clean":"2024-02-21T23:25:57.516216+0000","last_became_active":"2024-02-21T23:25:17.184574+0000","last_became_peered":"2024-02-21T23:25:17.184574+0000","last_unstale":"2024-02-21T23:25:57.516216+0000","last_undegraded":"2024-02-21T23:25:57.516216+0000","last_fullsized":"2024-02-21T23:25:57.516216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600404+0000","last_change":"2024-02-21T23:25:16.151031+0000","last_active":"2024-02-21T23:25:57.600404+0000","last_peered":"2024-02-21T23:25:57.600404+0000","last_clean":"2024-02-21T23:25:57.600404+0000","last_became_active":"2024-02-21T23:25:16.150767+0000","last_became_peered":"2024-02-21T23:25:16.150767+0000","last_unstale":"2024-02-21T23:25:57.600404+0000","last_undegraded":"2024-02-21T23:25:57.600404+0000","last_fullsized":"2024-02-21T23:25:57.600404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.515765+0000","last_change":"2024-02-21T23:25:21.683995+0000","last_active":"2024-02-21T23:25:57.515765+0000","last_peered":"2024-02-21T23:25:57.515765+0000","last_clean":"2024-02-21T23:25:57.515765+0000","last_became_active":"2024-02-21T23:25:21.683415+0000","last_became_peered":"2024-02-21T23:25:21.683415+0000","last_unstale":"2024-02-21T23:25:57.515765+0000","last_undegraded":"2024-02-21T23:25:57.515765+0000","last_fullsized":"2024-02-21T23:25:57.515765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519785+0000","last_change":"2024-02-21T23:25:17.187230+0000","last_active":"2024-02-21T23:25:57.519785+0000","last_peered":"2024-02-21T23:25:57.519785+0000","last_clean":"2024-02-21T23:25:57.519785+0000","last_became_active":"2024-02-21T23:25:17.186952+0000","last_became_peered":"2024-02-21T23:25:17.186952+0000","last_unstale":"2024-02-21T23:25:57.519785+0000","last_undegraded":"2024-02-21T23:25:57.519785+0000","last_fullsized":"2024-02-21T23:25:57.519785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524997+0000","last_change":"2024-02-21T23:25:16.134722+0000","last_active":"2024-02-21T23:25:57.524997+0000","last_peered":"2024-02-21T23:25:57.524997+0000","last_clean":"2024-02-21T23:25:57.524997+0000","last_became_active":"2024-02-21T23:25:16.134458+0000","last_became_peered":"2024-02-21T23:25:16.134458+0000","last_unstale":"2024-02-21T23:25:57.524997+0000","last_undegraded":"2024-02-21T23:25:57.524997+0000","last_fullsized":"2024-02-21T23:25:57.524997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519712+0000","last_change":"2024-02-21T23:25:21.684109+0000","last_active":"2024-02-21T23:25:57.519712+0000","last_peered":"2024-02-21T23:25:57.519712+0000","last_clean":"2024-02-21T23:25:57.519712+0000","last_became_active":"2024-02-21T23:25:21.683705+0000","last_became_peered":"2024-02-21T23:25:21.683705+0000","last_unstale":"2024-02-21T23:25:57.519712+0000","last_undegraded":"2024-02-21T23:25:57.519712+0000","last_fullsized":"2024-02-21T23:25:57.519712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524513+0000","last_change":"2024-02-21T23:25:17.188374+0000","last_active":"2024-02-21T23:25:57.524513+0000","last_peered":"2024-02-21T23:25:57.524513+0000","last_clean":"2024-02-21T23:25:57.524513+0000","last_became_active":"2024-02-21T23:25:17.187792+0000","last_became_peered":"2024-02-21T23:25:17.187792+0000","last_unstale":"2024-02-21T23:25:57.524513+0000","last_undegraded":"2024-02-21T23:25:57.524513+0000","last_fullsized":"2024-02-21T23:25:57.524513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.581970+0000","last_change":"2024-02-21T23:25:16.129446+0000","last_active":"2024-02-21T23:25:57.581970+0000","last_peered":"2024-02-21T23:25:57.581970+0000","last_clean":"2024-02-21T23:25:57.581970+0000","last_became_active":"2024-02-21T23:25:16.129164+0000","last_became_peered":"2024-02-21T23:25:16.129164+0000","last_unstale":"2024-02-21T23:25:57.581970+0000","last_undegraded":"2024-02-21T23:25:57.581970+0000","last_fullsized":"2024-02-21T23:25:57.581970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.602325+0000","last_change":"2024-02-21T23:25:57.600307+0000","last_active":"2024-02-21T23:25:57.602325+0000","last_peered":"2024-02-21T23:25:57.602325+0000","last_clean":"2024-02-21T23:25:57.602325+0000","last_became_active":"2024-02-21T23:25:57.599867+0000","last_became_peered":"2024-02-21T23:25:57.599867+0000","last_unstale":"2024-02-21T23:25:57.602325+0000","last_undegraded":"2024-02-21T23:25:57.602325+0000","last_fullsized":"2024-02-21T23:25:57.602325+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,5],"acting":[2,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518332+0000","last_change":"2024-02-21T23:25:22.248119+0000","last_active":"2024-02-21T23:25:57.518332+0000","last_peered":"2024-02-21T23:25:57.518332+0000","last_clean":"2024-02-21T23:25:57.518332+0000","last_became_active":"2024-02-21T23:25:22.247811+0000","last_became_peered":"2024-02-21T23:25:22.247811+0000","last_unstale":"2024-02-21T23:25:57.518332+0000","last_undegraded":"2024-02-21T23:25:57.518332+0000","last_fullsized":"2024-02-21T23:25:57.518332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599155+0000","last_change":"2024-02-21T23:25:17.189176+0000","last_active":"2024-02-21T23:25:57.599155+0000","last_peered":"2024-02-21T23:25:57.599155+0000","last_clean":"2024-02-21T23:25:57.599155+0000","last_became_active":"2024-02-21T23:25:17.188885+0000","last_became_peered":"2024-02-21T23:25:17.188885+0000","last_unstale":"2024-02-21T23:25:57.599155+0000","last_undegraded":"2024-02-21T23:25:57.599155+0000","last_fullsized":"2024-02-21T23:25:57.599155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523591+0000","last_change":"2024-02-21T23:25:16.132036+0000","last_active":"2024-02-21T23:25:57.523591+0000","last_peered":"2024-02-21T23:25:57.523591+0000","last_clean":"2024-02-21T23:25:57.523591+0000","last_became_active":"2024-02-21T23:25:16.131756+0000","last_became_peered":"2024-02-21T23:25:16.131756+0000","last_unstale":"2024-02-21T23:25:57.523591+0000","last_undegraded":"2024-02-21T23:25:57.523591+0000","last_fullsized":"2024-02-21T23:25:57.523591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.581794+0000","last_change":"2024-02-21T23:25:21.686133+0000","last_active":"2024-02-21T23:25:57.581794+0000","last_peered":"2024-02-21T23:25:57.581794+0000","last_clean":"2024-02-21T23:25:57.581794+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:25:57.581794+0000","last_undegraded":"2024-02-21T23:25:57.581794+0000","last_fullsized":"2024-02-21T23:25:57.581794+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519804+0000","last_change":"2024-02-21T23:25:17.135158+0000","last_active":"2024-02-21T23:25:57.519804+0000","last_peered":"2024-02-21T23:25:57.519804+0000","last_clean":"2024-02-21T23:25:57.519804+0000","last_became_active":"2024-02-21T23:25:17.134888+0000","last_became_peered":"2024-02-21T23:25:17.134888+0000","last_unstale":"2024-02-21T23:25:57.519804+0000","last_undegraded":"2024-02-21T23:25:57.519804+0000","last_fullsized":"2024-02-21T23:25:57.519804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518677+0000","last_change":"2024-02-21T23:25:16.116378+0000","last_active":"2024-02-21T23:25:57.518677+0000","last_peered":"2024-02-21T23:25:57.518677+0000","last_clean":"2024-02-21T23:25:57.518677+0000","last_became_active":"2024-02-21T23:25:16.116201+0000","last_became_peered":"2024-02-21T23:25:16.116201+0000","last_unstale":"2024-02-21T23:25:57.518677+0000","last_undegraded":"2024-02-21T23:25:57.518677+0000","last_fullsized":"2024-02-21T23:25:57.518677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.581847+0000","last_change":"2024-02-21T23:25:16.132314+0000","last_active":"2024-02-21T23:25:57.581847+0000","last_peered":"2024-02-21T23:25:57.581847+0000","last_clean":"2024-02-21T23:25:57.581847+0000","last_became_active":"2024-02-21T23:25:16.132062+0000","last_became_peered":"2024-02-21T23:25:16.132062+0000","last_unstale":"2024-02-21T23:25:57.581847+0000","last_undegraded":"2024-02-21T23:25:57.581847+0000","last_fullsized":"2024-02-21T23:25:57.581847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519386+0000","last_change":"2024-02-21T23:25:17.161451+0000","last_active":"2024-02-21T23:25:57.519386+0000","last_peered":"2024-02-21T23:25:57.519386+0000","last_clean":"2024-02-21T23:25:57.519386+0000","last_became_active":"2024-02-21T23:25:17.161176+0000","last_became_peered":"2024-02-21T23:25:17.161176+0000","last_unstale":"2024-02-21T23:25:57.519386+0000","last_undegraded":"2024-02-21T23:25:57.519386+0000","last_fullsized":"2024-02-21T23:25:57.519386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523342+0000","last_change":"2024-02-21T23:25:21.685724+0000","last_active":"2024-02-21T23:25:57.523342+0000","last_peered":"2024-02-21T23:25:57.523342+0000","last_clean":"2024-02-21T23:25:57.523342+0000","last_became_active":"2024-02-21T23:25:21.685375+0000","last_became_peered":"2024-02-21T23:25:21.685375+0000","last_unstale":"2024-02-21T23:25:57.523342+0000","last_undegraded":"2024-02-21T23:25:57.523342+0000","last_fullsized":"2024-02-21T23:25:57.523342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:56:28.795401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.520285+0000","last_change":"2024-02-21T23:25:16.134127+0000","last_active":"2024-02-21T23:25:57.520285+0000","last_peered":"2024-02-21T23:25:57.520285+0000","last_clean":"2024-02-21T23:25:57.520285+0000","last_became_active":"2024-02-21T23:25:16.133839+0000","last_became_peered":"2024-02-21T23:25:16.133839+0000","last_unstale":"2024-02-21T23:25:57.520285+0000","last_undegraded":"2024-02-21T23:25:57.520285+0000","last_fullsized":"2024-02-21T23:25:57.520285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.515224+0000","last_change":"2024-02-21T23:25:17.190455+0000","last_active":"2024-02-21T23:25:57.515224+0000","last_peered":"2024-02-21T23:25:57.515224+0000","last_clean":"2024-02-21T23:25:57.515224+0000","last_became_active":"2024-02-21T23:25:17.190194+0000","last_became_peered":"2024-02-21T23:25:17.190194+0000","last_unstale":"2024-02-21T23:25:57.515224+0000","last_undegraded":"2024-02-21T23:25:57.515224+0000","last_fullsized":"2024-02-21T23:25:57.515224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:03.451409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.526175+0000","last_change":"2024-02-21T23:25:57.526175+0000","last_active":"2024-02-21T23:25:57.526175+0000","last_peered":"2024-02-21T23:25:57.526175+0000","last_clean":"2024-02-21T23:25:57.526175+0000","last_became_active":"2024-02-21T23:25:57.525755+0000","last_became_peered":"2024-02-21T23:25:57.525755+0000","last_unstale":"2024-02-21T23:25:57.526175+0000","last_undegraded":"2024-02-21T23:25:57.526175+0000","last_fullsized":"2024-02-21T23:25:57.526175+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:18:17.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4,2],"acting":[3,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.581819+0000","last_change":"2024-02-21T23:25:17.138039+0000","last_active":"2024-02-21T23:25:57.581819+0000","last_peered":"2024-02-21T23:25:57.581819+0000","last_clean":"2024-02-21T23:25:57.581819+0000","last_became_active":"2024-02-21T23:25:17.137717+0000","last_became_peered":"2024-02-21T23:25:17.137717+0000","last_unstale":"2024-02-21T23:25:57.581819+0000","last_undegraded":"2024-02-21T23:25:57.581819+0000","last_fullsized":"2024-02-21T23:25:57.581819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524200+0000","last_change":"2024-02-21T23:25:17.194727+0000","last_active":"2024-02-21T23:25:57.524200+0000","last_peered":"2024-02-21T23:25:57.524200+0000","last_clean":"2024-02-21T23:25:57.524200+0000","last_became_active":"2024-02-21T23:25:17.194533+0000","last_became_peered":"2024-02-21T23:25:17.194533+0000","last_unstale":"2024-02-21T23:25:57.524200+0000","last_undegraded":"2024-02-21T23:25:57.524200+0000","last_fullsized":"2024-02-21T23:25:57.524200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523845+0000","last_change":"2024-02-21T23:25:17.191051+0000","last_active":"2024-02-21T23:25:57.523845+0000","last_peered":"2024-02-21T23:25:57.523845+0000","last_clean":"2024-02-21T23:25:57.523845+0000","last_became_active":"2024-02-21T23:25:17.190600+0000","last_became_peered":"2024-02-21T23:25:17.190600+0000","last_unstale":"2024-02-21T23:25:57.523845+0000","last_undegraded":"2024-02-21T23:25:57.523845+0000","last_fullsized":"2024-02-21T23:25:57.523845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519298+0000","last_change":"2024-02-21T23:25:17.135311+0000","last_active":"2024-02-21T23:25:57.519298+0000","last_peered":"2024-02-21T23:25:57.519298+0000","last_clean":"2024-02-21T23:25:57.519298+0000","last_became_active":"2024-02-21T23:25:17.135011+0000","last_became_peered":"2024-02-21T23:25:17.135011+0000","last_unstale":"2024-02-21T23:25:57.519298+0000","last_undegraded":"2024-02-21T23:25:57.519298+0000","last_fullsized":"2024-02-21T23:25:57.519298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.520712+0000","last_change":"2024-02-21T23:25:17.161696+0000","last_active":"2024-02-21T23:25:57.520712+0000","last_peered":"2024-02-21T23:25:57.520712+0000","last_clean":"2024-02-21T23:25:57.520712+0000","last_became_active":"2024-02-21T23:25:17.161401+0000","last_became_peered":"2024-02-21T23:25:17.161401+0000","last_unstale":"2024-02-21T23:25:57.520712+0000","last_undegraded":"2024-02-21T23:25:57.520712+0000","last_fullsized":"2024-02-21T23:25:57.520712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599904+0000","last_change":"2024-02-21T23:25:17.189731+0000","last_active":"2024-02-21T23:25:57.599904+0000","last_peered":"2024-02-21T23:25:57.599904+0000","last_clean":"2024-02-21T23:25:57.599904+0000","last_became_active":"2024-02-21T23:25:17.189484+0000","last_became_peered":"2024-02-21T23:25:17.189484+0000","last_unstale":"2024-02-21T23:25:57.599904+0000","last_undegraded":"2024-02-21T23:25:57.599904+0000","last_fullsized":"2024-02-21T23:25:57.599904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.582468+0000","last_change":"2024-02-21T23:25:17.184125+0000","last_active":"2024-02-21T23:25:57.582468+0000","last_peered":"2024-02-21T23:25:57.582468+0000","last_clean":"2024-02-21T23:25:57.582468+0000","last_became_active":"2024-02-21T23:25:17.183865+0000","last_became_peered":"2024-02-21T23:25:17.183865+0000","last_unstale":"2024-02-21T23:25:57.582468+0000","last_undegraded":"2024-02-21T23:25:57.582468+0000","last_fullsized":"2024-02-21T23:25:57.582468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.521137+0000","last_change":"2024-02-21T23:25:17.161482+0000","last_active":"2024-02-21T23:25:57.521137+0000","last_peered":"2024-02-21T23:25:57.521137+0000","last_clean":"2024-02-21T23:25:57.521137+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:25:57.521137+0000","last_undegraded":"2024-02-21T23:25:57.521137+0000","last_fullsized":"2024-02-21T23:25:57.521137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523480+0000","last_change":"2024-02-21T23:25:17.185691+0000","last_active":"2024-02-21T23:25:57.523480+0000","last_peered":"2024-02-21T23:25:57.523480+0000","last_clean":"2024-02-21T23:25:57.523480+0000","last_became_active":"2024-02-21T23:25:17.185420+0000","last_became_peered":"2024-02-21T23:25:17.185420+0000","last_unstale":"2024-02-21T23:25:57.523480+0000","last_undegraded":"2024-02-21T23:25:57.523480+0000","last_fullsized":"2024-02-21T23:25:57.523480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518760+0000","last_change":"2024-02-21T23:25:17.161321+0000","last_active":"2024-02-21T23:25:57.518760+0000","last_peered":"2024-02-21T23:25:57.518760+0000","last_clean":"2024-02-21T23:25:57.518760+0000","last_became_active":"2024-02-21T23:25:17.161010+0000","last_became_peered":"2024-02-21T23:25:17.161010+0000","last_unstale":"2024-02-21T23:25:57.518760+0000","last_undegraded":"2024-02-21T23:25:57.518760+0000","last_fullsized":"2024-02-21T23:25:57.518760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523764+0000","last_change":"2024-02-21T23:25:17.191072+0000","last_active":"2024-02-21T23:25:57.523764+0000","last_peered":"2024-02-21T23:25:57.523764+0000","last_clean":"2024-02-21T23:25:57.523764+0000","last_became_active":"2024-02-21T23:25:17.187741+0000","last_became_peered":"2024-02-21T23:25:17.187741+0000","last_unstale":"2024-02-21T23:25:57.523764+0000","last_undegraded":"2024-02-21T23:25:57.523764+0000","last_fullsized":"2024-02-21T23:25:57.523764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.519115+0000","last_change":"2024-02-21T23:25:17.186616+0000","last_active":"2024-02-21T23:25:57.519115+0000","last_peered":"2024-02-21T23:25:57.519115+0000","last_clean":"2024-02-21T23:25:57.519115+0000","last_became_active":"2024-02-21T23:25:17.186108+0000","last_became_peered":"2024-02-21T23:25:17.186108+0000","last_unstale":"2024-02-21T23:25:57.519115+0000","last_undegraded":"2024-02-21T23:25:57.519115+0000","last_fullsized":"2024-02-21T23:25:57.519115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.582197+0000","last_change":"2024-02-21T23:25:17.161428+0000","last_active":"2024-02-21T23:25:57.582197+0000","last_peered":"2024-02-21T23:25:57.582197+0000","last_clean":"2024-02-21T23:25:57.582197+0000","last_became_active":"2024-02-21T23:25:17.161140+0000","last_became_peered":"2024-02-21T23:25:17.161140+0000","last_unstale":"2024-02-21T23:25:57.582197+0000","last_undegraded":"2024-02-21T23:25:57.582197+0000","last_fullsized":"2024-02-21T23:25:57.582197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599180+0000","last_change":"2024-02-21T23:25:17.190666+0000","last_active":"2024-02-21T23:25:57.599180+0000","last_peered":"2024-02-21T23:25:57.599180+0000","last_clean":"2024-02-21T23:25:57.599180+0000","last_became_active":"2024-02-21T23:25:17.190420+0000","last_became_peered":"2024-02-21T23:25:17.190420+0000","last_unstale":"2024-02-21T23:25:57.599180+0000","last_undegraded":"2024-02-21T23:25:57.599180+0000","last_fullsized":"2024-02-21T23:25:57.599180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523140+0000","last_change":"2024-02-21T23:25:17.189583+0000","last_active":"2024-02-21T23:25:57.523140+0000","last_peered":"2024-02-21T23:25:57.523140+0000","last_clean":"2024-02-21T23:25:57.523140+0000","last_became_active":"2024-02-21T23:25:17.189302+0000","last_became_peered":"2024-02-21T23:25:17.189302+0000","last_unstale":"2024-02-21T23:25:57.523140+0000","last_undegraded":"2024-02-21T23:25:57.523140+0000","last_fullsized":"2024-02-21T23:25:57.523140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.524823+0000","last_change":"2024-02-21T23:25:17.188113+0000","last_active":"2024-02-21T23:25:57.524823+0000","last_peered":"2024-02-21T23:25:57.524823+0000","last_clean":"2024-02-21T23:25:57.524823+0000","last_became_active":"2024-02-21T23:25:17.187811+0000","last_became_peered":"2024-02-21T23:25:17.187811+0000","last_unstale":"2024-02-21T23:25:57.524823+0000","last_undegraded":"2024-02-21T23:25:57.524823+0000","last_fullsized":"2024-02-21T23:25:57.524823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.527915+0000","last_change":"2024-02-21T23:25:57.525755+0000","last_active":"2024-02-21T23:25:57.527915+0000","last_peered":"2024-02-21T23:25:57.527915+0000","last_clean":"2024-02-21T23:25:57.527915+0000","last_became_active":"2024-02-21T23:25:57.525443+0000","last_became_peered":"2024-02-21T23:25:57.525443+0000","last_unstale":"2024-02-21T23:25:57.527915+0000","last_undegraded":"2024-02-21T23:25:57.527915+0000","last_fullsized":"2024-02-21T23:25:57.527915+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599181+0000","last_change":"2024-02-21T23:25:17.186378+0000","last_active":"2024-02-21T23:25:57.599181+0000","last_peered":"2024-02-21T23:25:57.599181+0000","last_clean":"2024-02-21T23:25:57.599181+0000","last_became_active":"2024-02-21T23:25:17.186086+0000","last_became_peered":"2024-02-21T23:25:17.186086+0000","last_unstale":"2024-02-21T23:25:57.599181+0000","last_undegraded":"2024-02-21T23:25:57.599181+0000","last_fullsized":"2024-02-21T23:25:57.599181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.518718+0000","last_change":"2024-02-21T23:25:17.190934+0000","last_active":"2024-02-21T23:25:57.518718+0000","last_peered":"2024-02-21T23:25:57.518718+0000","last_clean":"2024-02-21T23:25:57.518718+0000","last_became_active":"2024-02-21T23:25:17.190680+0000","last_became_peered":"2024-02-21T23:25:17.190680+0000","last_unstale":"2024-02-21T23:25:57.518718+0000","last_undegraded":"2024-02-21T23:25:57.518718+0000","last_fullsized":"2024-02-21T23:25:57.518718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:22:04.238374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.520109+0000","last_change":"2024-02-21T23:25:17.135140+0000","last_active":"2024-02-21T23:25:57.520109+0000","last_peered":"2024-02-21T23:25:57.520109+0000","last_clean":"2024-02-21T23:25:57.520109+0000","last_became_active":"2024-02-21T23:25:17.134866+0000","last_became_peered":"2024-02-21T23:25:17.134866+0000","last_unstale":"2024-02-21T23:25:57.520109+0000","last_undegraded":"2024-02-21T23:25:57.520109+0000","last_fullsized":"2024-02-21T23:25:57.520109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.516213+0000","last_change":"2024-02-21T23:25:17.162204+0000","last_active":"2024-02-21T23:25:57.516213+0000","last_peered":"2024-02-21T23:25:57.516213+0000","last_clean":"2024-02-21T23:25:57.516213+0000","last_became_active":"2024-02-21T23:25:17.161807+0000","last_became_peered":"2024-02-21T23:25:17.161807+0000","last_unstale":"2024-02-21T23:25:57.516213+0000","last_undegraded":"2024-02-21T23:25:57.516213+0000","last_fullsized":"2024-02-21T23:25:57.516213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.582025+0000","last_change":"2024-02-21T23:25:17.188932+0000","last_active":"2024-02-21T23:25:57.582025+0000","last_peered":"2024-02-21T23:25:57.582025+0000","last_clean":"2024-02-21T23:25:57.582025+0000","last_became_active":"2024-02-21T23:25:17.188646+0000","last_became_peered":"2024-02-21T23:25:17.188646+0000","last_unstale":"2024-02-21T23:25:57.582025+0000","last_undegraded":"2024-02-21T23:25:57.582025+0000","last_fullsized":"2024-02-21T23:25:57.582025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.582424+0000","last_change":"2024-02-21T23:25:17.188333+0000","last_active":"2024-02-21T23:25:57.582424+0000","last_peered":"2024-02-21T23:25:57.582424+0000","last_clean":"2024-02-21T23:25:57.582424+0000","last_became_active":"2024-02-21T23:25:17.188066+0000","last_became_peered":"2024-02-21T23:25:17.188066+0000","last_unstale":"2024-02-21T23:25:57.582424+0000","last_undegraded":"2024-02-21T23:25:57.582424+0000","last_fullsized":"2024-02-21T23:25:57.582424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599302+0000","last_change":"2024-02-21T23:25:17.187549+0000","last_active":"2024-02-21T23:25:57.599302+0000","last_peered":"2024-02-21T23:25:57.599302+0000","last_clean":"2024-02-21T23:25:57.599302+0000","last_became_active":"2024-02-21T23:25:17.187297+0000","last_became_peered":"2024-02-21T23:25:17.187297+0000","last_unstale":"2024-02-21T23:25:57.599302+0000","last_undegraded":"2024-02-21T23:25:57.599302+0000","last_fullsized":"2024-02-21T23:25:57.599302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.530624+0000","last_change":"2024-02-21T23:25:57.526373+0000","last_active":"2024-02-21T23:25:57.530624+0000","last_peered":"2024-02-21T23:25:57.530624+0000","last_clean":"2024-02-21T23:25:57.530624+0000","last_became_active":"2024-02-21T23:25:57.526068+0000","last_became_peered":"2024-02-21T23:25:57.526068+0000","last_unstale":"2024-02-21T23:25:57.530624+0000","last_undegraded":"2024-02-21T23:25:57.530624+0000","last_fullsized":"2024-02-21T23:25:57.530624+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.598812+0000","last_change":"2024-02-21T23:25:17.183264+0000","last_active":"2024-02-21T23:25:57.598812+0000","last_peered":"2024-02-21T23:25:57.598812+0000","last_clean":"2024-02-21T23:25:57.598812+0000","last_became_active":"2024-02-21T23:25:17.182924+0000","last_became_peered":"2024-02-21T23:25:17.182924+0000","last_unstale":"2024-02-21T23:25:57.598812+0000","last_undegraded":"2024-02-21T23:25:57.598812+0000","last_fullsized":"2024-02-21T23:25:57.598812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.523762+0000","last_change":"2024-02-21T23:25:17.190841+0000","last_active":"2024-02-21T23:25:57.523762+0000","last_peered":"2024-02-21T23:25:57.523762+0000","last_clean":"2024-02-21T23:25:57.523762+0000","last_became_active":"2024-02-21T23:25:17.190276+0000","last_became_peered":"2024-02-21T23:25:17.190276+0000","last_unstale":"2024-02-21T23:25:57.523762+0000","last_undegraded":"2024-02-21T23:25:57.523762+0000","last_fullsized":"2024-02-21T23:25:57.523762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.598781+0000","last_change":"2024-02-21T23:25:17.189872+0000","last_active":"2024-02-21T23:25:57.598781+0000","last_peered":"2024-02-21T23:25:57.598781+0000","last_clean":"2024-02-21T23:25:57.598781+0000","last_became_active":"2024-02-21T23:25:17.189614+0000","last_became_peered":"2024-02-21T23:25:17.189614+0000","last_unstale":"2024-02-21T23:25:57.598781+0000","last_undegraded":"2024-02-21T23:25:57.598781+0000","last_fullsized":"2024-02-21T23:25:57.598781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600458+0000","last_change":"2024-02-21T23:25:57.600458+0000","last_active":"2024-02-21T23:25:57.600458+0000","last_peered":"2024-02-21T23:25:57.600458+0000","last_clean":"2024-02-21T23:25:57.600458+0000","last_became_active":"2024-02-21T23:25:57.600094+0000","last_became_peered":"2024-02-21T23:25:57.600094+0000","last_unstale":"2024-02-21T23:25:57.600458+0000","last_undegraded":"2024-02-21T23:25:57.600458+0000","last_fullsized":"2024-02-21T23:25:57.600458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:20:16.177299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.517854+0000","last_change":"2024-02-21T23:25:17.191159+0000","last_active":"2024-02-21T23:25:57.517854+0000","last_peered":"2024-02-21T23:25:57.517854+0000","last_clean":"2024-02-21T23:25:57.517854+0000","last_became_active":"2024-02-21T23:25:17.190576+0000","last_became_peered":"2024-02-21T23:25:17.190576+0000","last_unstale":"2024-02-21T23:25:57.517854+0000","last_undegraded":"2024-02-21T23:25:57.517854+0000","last_fullsized":"2024-02-21T23:25:57.517854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.600143+0000","last_change":"2024-02-21T23:25:17.189287+0000","last_active":"2024-02-21T23:25:57.600143+0000","last_peered":"2024-02-21T23:25:57.600143+0000","last_clean":"2024-02-21T23:25:57.600143+0000","last_became_active":"2024-02-21T23:25:17.189029+0000","last_became_peered":"2024-02-21T23:25:17.189029+0000","last_unstale":"2024-02-21T23:25:57.600143+0000","last_undegraded":"2024-02-21T23:25:57.600143+0000","last_fullsized":"2024-02-21T23:25:57.600143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:25:57.599103+0000","last_change":"2024-02-21T23:25:17.183737+0000","last_active":"2024-02-21T23:25:57.599103+0000","last_peered":"2024-02-21T23:25:57.599103+0000","last_clean":"2024-02-21T23:25:57.599103+0000","last_became_active":"2024-02-21T23:25:17.183410+0000","last_became_peered":"2024-02-21T23:25:17.183410+0000","last_unstale":"2024-02-21T23:25:57.599103+0000","last_undegraded":"2024-02-21T23:25:57.599103+0000","last_fullsized":"2024-02-21T23:25:57.599103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":705580,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":118472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":114720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":114756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":111410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":114630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":114790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":705636,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-21T23:25:58.869 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-02-21T23:25:58.869 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-21T23:25:58.869 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-02-21T23:25:58.869 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-02-21T23:25:59.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.017+0000 7f949f59c640 1 Processor -- start 2024-02-21T23:25:59.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.018+0000 7f949f59c640 1 -- start start 2024-02-21T23:25:59.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.018+0000 7f949f59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 0x7f949806d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.018+0000 7f949f59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f94980b0fe0 0x7f9498072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.018+0000 7f949f59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9498072f80 0x7f949806de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.018+0000 7f949f59c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949815afd0 con 0x7f9498072f80 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.018+0000 7f949f59c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949815a170 con 0x7f94980b0fe0 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.018+0000 7f949f59c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949815a450 con 0x7f9498154300 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.019+0000 7f949db39640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9498072f80 0x7f949806de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.019+0000 7f949d338640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 0x7f949806d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.019+0000 7f949cb37640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f94980b0fe0 0x7f9498072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.019+0000 7f949cb37640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f94980b0fe0 0x7f9498072a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42894/0 (socket says 172.21.15.133:42894) 2024-02-21T23:25:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.019+0000 7f949cb37640 1 -- 172.21.15.133:0/161758916 learned_addr learned my addr 172.21.15.133:0/161758916 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:59.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.022+0000 7f949db39640 1 -- 172.21.15.133:0/161758916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 msgr2=0x7f949806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:59.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.022+0000 7f949db39640 1 --2- 172.21.15.133:0/161758916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 0x7f949806d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.022+0000 7f949db39640 1 -- 172.21.15.133:0/161758916 >> v2:172.21.15.142:3300/0 conn(0x7f94980b0fe0 msgr2=0x7f9498072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:59.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.022+0000 7f949db39640 1 --2- 172.21.15.133:0/161758916 >> v2:172.21.15.142:3300/0 conn(0x7f94980b0fe0 0x7f9498072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.022+0000 7f949db39640 1 -- 172.21.15.133:0/161758916 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949815a730 con 0x7f9498072f80 2024-02-21T23:25:59.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.024+0000 7f949db39640 1 --2- 172.21.15.133:0/161758916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9498072f80 0x7f949806de00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f94980619a0 tx=0x7f949002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cc6666917dcc27 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:59.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.027+0000 7f948ffff640 1 -- 172.21.15.133:0/161758916 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490039070 con 0x7f9498072f80 2024-02-21T23:25:59.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.027+0000 7f948ffff640 1 -- 172.21.15.133:0/161758916 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949002cc60 con 0x7f9498072f80 2024-02-21T23:25:59.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.027+0000 7f948ffff640 1 -- 172.21.15.133:0/161758916 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490034660 con 0x7f9498072f80 2024-02-21T23:25:59.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.028+0000 7f949f59c640 1 -- 172.21.15.133:0/161758916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9498072f80 msgr2=0x7f949806de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:59.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.028+0000 7f949f59c640 1 --2- 172.21.15.133:0/161758916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9498072f80 0x7f949806de00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f94980619a0 tx=0x7f949002beb0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.028+0000 7f949f59c640 1 -- 172.21.15.133:0/161758916 shutdown_connections 2024-02-21T23:25:59.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.028+0000 7f949f59c640 1 --2- 172.21.15.133:0/161758916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9498072f80 0x7f949806de00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.028+0000 7f949f59c640 1 --2- 172.21.15.133:0/161758916 >> v2:172.21.15.142:3300/0 conn(0x7f94980b0fe0 0x7f9498072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.028+0000 7f949f59c640 1 --2- 172.21.15.133:0/161758916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 0x7f949806d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.028+0000 7f949f59c640 1 -- 172.21.15.133:0/161758916 >> 172.21.15.133:0/161758916 conn(0x7f94980cc620 msgr2=0x7f94980c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:59.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.028+0000 7f949f59c640 1 -- 172.21.15.133:0/161758916 shutdown_connections 2024-02-21T23:25:59.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.028+0000 7f949f59c640 1 -- 172.21.15.133:0/161758916 wait complete. 2024-02-21T23:25:59.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.030+0000 7f949f59c640 1 Processor -- start 2024-02-21T23:25:59.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.030+0000 7f949f59c640 1 -- start start 2024-02-21T23:25:59.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.030+0000 7f949f59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9498072f80 0x7f9498153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.030+0000 7f949f59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94980b0fe0 0x7f94981f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.030+0000 7f949f59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 0x7f949814bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.030+0000 7f949f59c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94981546e0 con 0x7f94980b0fe0 2024-02-21T23:25:59.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.030+0000 7f949f59c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949815acf0 con 0x7f9498072f80 2024-02-21T23:25:59.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.030+0000 7f949f59c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949806d5e0 con 0x7f9498154300 2024-02-21T23:25:59.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.031+0000 7f949d338640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9498072f80 0x7f9498153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.031+0000 7f949db39640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 0x7f949814bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.031+0000 7f949d338640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9498072f80 0x7f9498153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42910/0 (socket says 172.21.15.133:42910) 2024-02-21T23:25:59.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.031+0000 7f949d338640 1 -- 172.21.15.133:0/89445740 learned_addr learned my addr 172.21.15.133:0/89445740 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:59.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.031+0000 7f949cb37640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94980b0fe0 0x7f94981f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.035+0000 7f949cb37640 1 -- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 msgr2=0x7f949814bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:59.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.035+0000 7f949cb37640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 0x7f949814bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.035+0000 7f949cb37640 1 -- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9498072f80 msgr2=0x7f9498153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:59.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.035+0000 7f949cb37640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9498072f80 0x7f9498153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.035+0000 7f949cb37640 1 -- 172.21.15.133:0/89445740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498153990 con 0x7f94980b0fe0 2024-02-21T23:25:59.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.037+0000 7f949cb37640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94980b0fe0 0x7f94981f42b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f948800e800 tx=0x7f948800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:59.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.039+0000 7f948dffb640 1 -- 172.21.15.133:0/89445740 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488017070 con 0x7f94980b0fe0 2024-02-21T23:25:59.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.039+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94981f47f0 con 0x7f94980b0fe0 2024-02-21T23:25:59.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.039+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94981f0570 con 0x7f94980b0fe0 2024-02-21T23:25:59.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.041+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949806c670 con 0x7f94980b0fe0 2024-02-21T23:25:59.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.041+0000 7f948dffb640 1 -- 172.21.15.133:0/89445740 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9488005990 con 0x7f94980b0fe0 2024-02-21T23:25:59.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.041+0000 7f948dffb640 1 -- 172.21.15.133:0/89445740 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488021060 con 0x7f94980b0fe0 2024-02-21T23:25:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.291+0000 7f948dffb640 1 -- 172.21.15.133:0/89445740 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f948801a070 con 0x7f94980b0fe0 2024-02-21T23:25:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.291+0000 7f948dffb640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f947c06b4d0 0x7f947c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.292+0000 7f949d338640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f947c06b4d0 0x7f947c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.292+0000 7f949d338640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f947c06b4d0 0x7f947c06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9494009b60 tx=0x7f9494009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:59.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.307+0000 7f948dffb640 1 -- 172.21.15.133:0/89445740 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f9488056cd0 con 0x7f94980b0fe0 2024-02-21T23:25:59.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.307+0000 7f948dffb640 1 -- 172.21.15.133:0/89445740 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f9488058c30 con 0x7f94980b0fe0 2024-02-21T23:25:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.522+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f949806a130 con 0x7f94980b0fe0 2024-02-21T23:25:59.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.526+0000 7f948dffb640 1 -- 172.21.15.133:0/89445740 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f948805ccd0 con 0x7f94980b0fe0 2024-02-21T23:25:59.512 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:25:59.512 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-21T23:25:59.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f947c06b4d0 msgr2=0x7f947c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:59.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f947c06b4d0 0x7f947c06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9494009b60 tx=0x7f9494009190 comp rx=0 tx=0).stop 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94980b0fe0 msgr2=0x7f94981f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94980b0fe0 0x7f94981f42b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f948800e800 tx=0x7f948800ecc0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 shutdown_connections 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f947c06b4d0 0x7f947c06d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9498154300 0x7f949814bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94980b0fe0 0x7f94981f42b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 --2- 172.21.15.133:0/89445740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9498072f80 0x7f9498153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 >> 172.21.15.133:0/89445740 conn(0x7f94980cc620 msgr2=0x7f949806bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.528+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 shutdown_connections 2024-02-21T23:25:59.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.529+0000 7f949f59c640 1 -- 172.21.15.133:0/89445740 wait complete. 2024-02-21T23:25:59.525 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-02-21T23:25:59.526 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:25:59.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.640+0000 7f25efad5640 1 Processor -- start 2024-02-21T23:25:59.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.640+0000 7f25efad5640 1 -- start start 2024-02-21T23:25:59.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.641+0000 7f25efad5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f25e80cfe10 0x7f25e80d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.641+0000 7f25efad5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cc890 0x7f25e80ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.641+0000 7f25efad5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e80cc100 con 0x7f25e815a170 2024-02-21T23:25:59.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.641+0000 7f25efad5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e80cc400 con 0x7f25e80cfe10 2024-02-21T23:25:59.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.641+0000 7f25efad5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80cd240 con 0x7f25e80cc890 2024-02-21T23:25:59.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.642+0000 7f25ee072640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cc890 0x7f25e80ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.642+0000 7f25ed871640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f25e80cfe10 0x7f25e80d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.642+0000 7f25ee072640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cc890 0x7f25e80ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:46114/0 (socket says 172.21.15.133:46114) 2024-02-21T23:25:59.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.642+0000 7f25ee072640 1 -- 172.21.15.133:0/4009480894 learned_addr learned my addr 172.21.15.133:0/4009480894 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:59.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.645+0000 7f25ee072640 1 -- 172.21.15.133:0/4009480894 >> v2:172.21.15.142:3300/0 conn(0x7f25e80cfe10 msgr2=0x7f25e80d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:25:59.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.645+0000 7f25ee072640 1 --2- 172.21.15.133:0/4009480894 >> v2:172.21.15.142:3300/0 conn(0x7f25e80cfe10 0x7f25e80d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.645+0000 7f25ee072640 1 -- 172.21.15.133:0/4009480894 >> v1:172.21.15.142:6789/0 conn(0x7f25e815a170 legacy=0x7f25e80ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:59.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.645+0000 7f25ee072640 1 -- 172.21.15.133:0/4009480894 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e815a6b0 con 0x7f25e80cc890 2024-02-21T23:25:59.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.647+0000 7f25ee072640 1 --2- 172.21.15.133:0/4009480894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cc890 0x7f25e80ccd00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009e50 tx=0x7f25e002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b2c19c56c5eaa server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:59.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.650+0000 7f25ec86f640 1 -- 172.21.15.133:0/4009480894 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e002ca70 con 0x7f25e80cc890 2024-02-21T23:25:59.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.650+0000 7f25ec86f640 1 -- 172.21.15.133:0/4009480894 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e002cc10 con 0x7f25e80cc890 2024-02-21T23:25:59.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.650+0000 7f25ec86f640 1 -- 172.21.15.133:0/4009480894 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e00343f0 con 0x7f25e80cc890 2024-02-21T23:25:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.651+0000 7f25efad5640 1 -- 172.21.15.133:0/4009480894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cc890 msgr2=0x7f25e80ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.651+0000 7f25efad5640 1 --2- 172.21.15.133:0/4009480894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cc890 0x7f25e80ccd00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009e50 tx=0x7f25e002bed0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.651+0000 7f25efad5640 1 -- 172.21.15.133:0/4009480894 shutdown_connections 2024-02-21T23:25:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.651+0000 7f25efad5640 1 --2- 172.21.15.133:0/4009480894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cc890 0x7f25e80ccd00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.651+0000 7f25efad5640 1 --2- 172.21.15.133:0/4009480894 >> v2:172.21.15.142:3300/0 conn(0x7f25e80cfe10 0x7f25e80d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.651+0000 7f25efad5640 1 -- 172.21.15.133:0/4009480894 >> 172.21.15.133:0/4009480894 conn(0x7f25e8061b50 msgr2=0x7f25e8061f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:25:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.651+0000 7f25efad5640 1 -- 172.21.15.133:0/4009480894 shutdown_connections 2024-02-21T23:25:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.651+0000 7f25efad5640 1 -- 172.21.15.133:0/4009480894 wait complete. 2024-02-21T23:25:59.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.652+0000 7f25efad5640 1 Processor -- start 2024-02-21T23:25:59.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.652+0000 7f25efad5640 1 -- start start 2024-02-21T23:25:59.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.653+0000 7f25efad5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e80cc890 0x7f25e80c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.653+0000 7f25efad5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cfe10 0x7f25e8153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.653+0000 7f25efad5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25e815a170 0x7f25e81f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.653+0000 7f25efad5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815af30 con 0x7f25e815a170 2024-02-21T23:25:59.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.653+0000 7f25efad5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815ac50 con 0x7f25e80cc890 2024-02-21T23:25:59.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.653+0000 7f25efad5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80cab50 con 0x7f25e80cfe10 2024-02-21T23:25:59.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.653+0000 7f25ed070640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cfe10 0x7f25e8153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.653+0000 7f25ed871640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e80cc890 0x7f25e80c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.653+0000 7f25ee072640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25e815a170 0x7f25e81f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.654+0000 7f25ed070640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cfe10 0x7f25e8153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:46124/0 (socket says 172.21.15.133:46124) 2024-02-21T23:25:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.654+0000 7f25ed070640 1 -- 172.21.15.133:0/3127729650 learned_addr learned my addr 172.21.15.133:0/3127729650 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:25:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.657+0000 7f25ed070640 1 -- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e80cc890 msgr2=0x7f25e80c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.657+0000 7f25ed070640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e80cc890 0x7f25e80c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.657+0000 7f25ed070640 1 -- 172.21.15.133:0/3127729650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25e815a170 msgr2=0x7f25e81f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:25:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.657+0000 7f25ed070640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25e815a170 0x7f25e81f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:25:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.657+0000 7f25ed070640 1 -- 172.21.15.133:0/3127729650 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e8146e20 con 0x7f25e80cfe10 2024-02-21T23:25:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.659+0000 7f25ed070640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cfe10 0x7f25e8153920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c3f0 tx=0x7f25d800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:59.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.663+0000 7f25de7fc640 1 -- 172.21.15.133:0/3127729650 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80052d0 con 0x7f25e80cfe10 2024-02-21T23:25:59.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.663+0000 7f25de7fc640 1 -- 172.21.15.133:0/3127729650 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d8005490 con 0x7f25e80cfe10 2024-02-21T23:25:59.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.663+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e80c29c0 con 0x7f25e80cfe10 2024-02-21T23:25:59.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.663+0000 7f25de7fc640 1 -- 172.21.15.133:0/3127729650 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8007260 con 0x7f25e80cfe10 2024-02-21T23:25:59.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.663+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e80c7f10 con 0x7f25e80cfe10 2024-02-21T23:25:59.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.664+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e80cb5e0 con 0x7f25e80cfe10 2024-02-21T23:25:59.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.682+0000 7f25de7fc640 1 -- 172.21.15.133:0/3127729650 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25d8007400 con 0x7f25e80cfe10 2024-02-21T23:25:59.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.683+0000 7f25de7fc640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f25cc06b520 0x7f25cc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:25:59.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.683+0000 7f25ed871640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f25cc06b520 0x7f25cc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:25:59.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.683+0000 7f25ed871640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f25cc06b520 0x7f25cc06d9e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009b20 tx=0x7f25e4009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:25:59.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.685+0000 7f25de7fc640 1 -- 172.21.15.133:0/3127729650 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f25d8056720 con 0x7f25e80cfe10 2024-02-21T23:25:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:25:59.929+0000 7f25de7fc640 1 -- 172.21.15.133:0/3127729650 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f25e80cb5e0 con 0x7f25e80cfe10 2024-02-21T23:26:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.104+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25e8073080 con 0x7f25e80cfe10 2024-02-21T23:26:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.149+0000 7f25de7fc640 1 -- 172.21.15.133:0/3127729650 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19091 (secure 0 0 0) 0x7f25e8073080 con 0x7f25e80cfe10 2024-02-21T23:26:00.136 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:26:00.136 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:57.029812+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":0}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:26:00.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f25cc06b520 msgr2=0x7f25cc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f25cc06b520 0x7f25cc06d9e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009b20 tx=0x7f25e4009190 comp rx=0 tx=0).stop 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cfe10 msgr2=0x7f25e8153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cfe10 0x7f25e8153920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c3f0 tx=0x7f25d800c8b0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 shutdown_connections 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f25cc06b520 0x7f25cc06d9e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25e815a170 0x7f25e81f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e80cfe10 0x7f25e8153920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 --2- 172.21.15.133:0/3127729650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e80cc890 0x7f25e80c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 >> 172.21.15.133:0/3127729650 conn(0x7f25e8061b50 msgr2=0x7f25e814dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.151+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 shutdown_connections 2024-02-21T23:26:00.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.152+0000 7f25efad5640 1 -- 172.21.15.133:0/3127729650 wait complete. 2024-02-21T23:26:00.149 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-21T23:25:02.902698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-21T23:25:13.993803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-21T23:25:15.604077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-21T23:25:19.468744+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-21T23:26:00.149 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-21T23:26:00.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.267+0000 7f32dff1e640 1 Processor -- start 2024-02-21T23:26:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.267+0000 7f32dff1e640 1 -- start start 2024-02-21T23:26:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.268+0000 7f32dff1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cfe10 0x7f32d80d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.268+0000 7f32dff1e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d80cc100 con 0x7f32d80cc890 2024-02-21T23:26:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.268+0000 7f32dff1e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d815a6b0 con 0x7f32d815a170 2024-02-21T23:26:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.268+0000 7f32dff1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80cc400 con 0x7f32d80cfe10 2024-02-21T23:26:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.268+0000 7f32de4bb640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f32d80cc890 0x7f32d80ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51468/0 (socket says 172.21.15.133:51468) 2024-02-21T23:26:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.268+0000 7f32ddcba640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cfe10 0x7f32d80d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.268+0000 7f32de4bb640 1 -- 172.21.15.133:0/1756815624 learned_addr learned my addr 172.21.15.133:0/1756815624 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:26:00.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.271+0000 7f32dccb8640 1 -- 172.21.15.133:0/1756815624 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106970395 0 0) 0x7f32d80cc100 con 0x7f32d80cc890 2024-02-21T23:26:00.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.271+0000 7f32dccb8640 1 -- 172.21.15.133:0/1756815624 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32c4003520 con 0x7f32d80cc890 2024-02-21T23:26:00.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.271+0000 7f32ddcba640 1 -- 172.21.15.133:0/1756815624 >> v1:172.21.15.142:6789/0 conn(0x7f32d815a170 legacy=0x7f32d80ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.271+0000 7f32ddcba640 1 -- 172.21.15.133:0/1756815624 >> v1:172.21.15.133:6789/0 conn(0x7f32d80cc890 legacy=0x7f32d80ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.271+0000 7f32ddcba640 1 -- 172.21.15.133:0/1756815624 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d815a970 con 0x7f32d80cfe10 2024-02-21T23:26:00.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.273+0000 7f32ddcba640 1 --2- 172.21.15.133:0/1756815624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cfe10 0x7f32d80d01f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32d40097d0 tx=0x7f32d402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8c147e22fb1977 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:00.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dccb8640 1 -- 172.21.15.133:0/1756815624 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4035020 con 0x7f32d80cfe10 2024-02-21T23:26:00.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dccb8640 1 -- 172.21.15.133:0/1756815624 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d4031800 con 0x7f32d80cfe10 2024-02-21T23:26:00.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dccb8640 1 -- 172.21.15.133:0/1756815624 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4031bd0 con 0x7f32d80cfe10 2024-02-21T23:26:00.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dff1e640 1 -- 172.21.15.133:0/1756815624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cfe10 msgr2=0x7f32d80d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dff1e640 1 --2- 172.21.15.133:0/1756815624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cfe10 0x7f32d80d01f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32d40097d0 tx=0x7f32d402b8a0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dff1e640 1 -- 172.21.15.133:0/1756815624 shutdown_connections 2024-02-21T23:26:00.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dff1e640 1 --2- 172.21.15.133:0/1756815624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cfe10 0x7f32d80d01f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dff1e640 1 -- 172.21.15.133:0/1756815624 >> 172.21.15.133:0/1756815624 conn(0x7f32d8061b50 msgr2=0x7f32d8061f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:00.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dff1e640 1 -- 172.21.15.133:0/1756815624 shutdown_connections 2024-02-21T23:26:00.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.277+0000 7f32dff1e640 1 -- 172.21.15.133:0/1756815624 wait complete. 2024-02-21T23:26:00.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.279+0000 7f32dff1e640 1 Processor -- start 2024-02-21T23:26:00.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.279+0000 7f32dff1e640 1 -- start start 2024-02-21T23:26:00.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.280+0000 7f32dff1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cc890 0x7f32d81f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.280+0000 7f32dff1e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d80cfe10 0x7f32d81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.280+0000 7f32dff1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d815a170 0x7f32d80ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.280+0000 7f32dff1e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80cab50 con 0x7f32d815a170 2024-02-21T23:26:00.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.280+0000 7f32dff1e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d815af30 con 0x7f32d80cfe10 2024-02-21T23:26:00.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.280+0000 7f32dff1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80ce1a0 con 0x7f32d80cc890 2024-02-21T23:26:00.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.280+0000 7f32de4bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d815a170 0x7f32d80ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.280+0000 7f32de4bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d815a170 0x7f32d80ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40310/0 (socket says 172.21.15.133:40310) 2024-02-21T23:26:00.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.280+0000 7f32de4bb640 1 -- 172.21.15.133:0/2239212357 learned_addr learned my addr 172.21.15.133:0/2239212357 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:00.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.281+0000 7f32ddcba640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cc890 0x7f32d81f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.281+0000 7f32dd4b9640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d80cfe10 0x7f32d81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.285+0000 7f32ddcba640 1 -- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d80cfe10 msgr2=0x7f32d81527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.285+0000 7f32ddcba640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d80cfe10 0x7f32d81527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.285+0000 7f32ddcba640 1 -- 172.21.15.133:0/2239212357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d815a170 msgr2=0x7f32d80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.285+0000 7f32ddcba640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d815a170 0x7f32d80ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.285+0000 7f32ddcba640 1 -- 172.21.15.133:0/2239212357 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d80c7630 con 0x7f32d80cc890 2024-02-21T23:26:00.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.287+0000 7f32ddcba640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cc890 0x7f32d81f3c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32d4033000 tx=0x7f32d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:00.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.291+0000 7f32ce7fc640 1 -- 172.21.15.133:0/2239212357 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4041070 con 0x7f32d80cc890 2024-02-21T23:26:00.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.291+0000 7f32ce7fc640 1 -- 172.21.15.133:0/2239212357 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d403ad00 con 0x7f32d80cc890 2024-02-21T23:26:00.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.291+0000 7f32ce7fc640 1 -- 172.21.15.133:0/2239212357 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4035040 con 0x7f32d80cc890 2024-02-21T23:26:00.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.291+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d80c7910 con 0x7f32d80cc890 2024-02-21T23:26:00.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.291+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d81f01c0 con 0x7f32d80cc890 2024-02-21T23:26:00.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.292+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d8073080 con 0x7f32d80cc890 2024-02-21T23:26:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.549+0000 7f32ce7fc640 1 -- 172.21.15.133:0/2239212357 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32d404c060 con 0x7f32d80cc890 2024-02-21T23:26:00.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.550+0000 7f32ce7fc640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f32bc06b520 0x7f32bc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.550+0000 7f32dd4b9640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f32bc06b520 0x7f32bc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.551+0000 7f32dd4b9640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f32bc06b520 0x7f32bc06d9e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32d80c9ea0 tx=0x7f32c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:00.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.553+0000 7f32ce7fc640 1 -- 172.21.15.133:0/2239212357 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f32d40788f0 con 0x7f32d80cc890 2024-02-21T23:26:00.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.564+0000 7f32ce7fc640 1 -- 172.21.15.133:0/2239212357 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f32d8073080 con 0x7f32d80cc890 2024-02-21T23:26:00.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.742+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32d81efcd0 con 0x7f32d80cc890 2024-02-21T23:26:00.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.750+0000 7f32ce7fc640 1 -- 172.21.15.133:0/2239212357 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32d407a850 con 0x7f32d80cc890 2024-02-21T23:26:00.735 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f32bc06b520 msgr2=0x7f32bc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f32bc06b520 0x7f32bc06d9e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32d80c9ea0 tx=0x7f32c8009040 comp rx=0 tx=0).stop 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cc890 msgr2=0x7f32d81f3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cc890 0x7f32d81f3c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32d4033000 tx=0x7f32d40046c0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 shutdown_connections 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f32bc06b520 0x7f32bc06d9e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d815a170 0x7f32d80ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d80cfe10 0x7f32d81527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 --2- 172.21.15.133:0/2239212357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d80cc890 0x7f32d81f3c50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 >> 172.21.15.133:0/2239212357 conn(0x7f32d8061b50 msgr2=0x7f32d80740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 shutdown_connections 2024-02-21T23:26:00.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.752+0000 7f32dff1e640 1 -- 172.21.15.133:0/2239212357 wait complete. 2024-02-21T23:26:00.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-21T23:26:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.866+0000 7fe1cccc8640 1 Processor -- start 2024-02-21T23:26:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.866+0000 7fe1cccc8640 1 -- start start 2024-02-21T23:26:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.867+0000 7fe1cccc8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 0x7fe1c80c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.867+0000 7fe1cccc8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1c80cd900 0x7fe1c80cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.867+0000 7fe1cccc8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80c3760 con 0x7fe1c80c4600 2024-02-21T23:26:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.867+0000 7fe1cccc8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c80c3a00 con 0x7fe1c80cd900 2024-02-21T23:26:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.867+0000 7fe1cccc8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80c3ce0 con 0x7fe1c8154120 2024-02-21T23:26:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.868+0000 7fe1c6d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1c80cd900 0x7fe1c80cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.868+0000 7fe1c659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 0x7fe1c80c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.868+0000 7fe1c6d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1c80cd900 0x7fe1c80cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42940/0 (socket says 172.21.15.133:42940) 2024-02-21T23:26:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.868+0000 7fe1c6d9d640 1 -- 172.21.15.133:0/1322701081 learned_addr learned my addr 172.21.15.133:0/1322701081 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.870+0000 7fe1c6d9d640 1 -- 172.21.15.133:0/1322701081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 msgr2=0x7fe1c80c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.870+0000 7fe1c6d9d640 1 --2- 172.21.15.133:0/1322701081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 0x7fe1c80c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.870+0000 7fe1c6d9d640 1 -- 172.21.15.133:0/1322701081 >> v1:172.21.15.133:6789/0 conn(0x7fe1c80c4600 legacy=0x7fe1c815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.870+0000 7fe1c6d9d640 1 -- 172.21.15.133:0/1322701081 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80c3fc0 con 0x7fe1c80cd900 2024-02-21T23:26:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.872+0000 7fe1c6d9d640 1 --2- 172.21.15.133:0/1322701081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80cd900 0x7fe1c80cdd70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009e50 tx=0x7fe1c002bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1de01c3851650a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.876+0000 7fe1c559a640 1 -- 172.21.15.133:0/1322701081 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c002cb90 con 0x7fe1c80cd900 2024-02-21T23:26:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.876+0000 7fe1c559a640 1 -- 172.21.15.133:0/1322701081 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c002cd30 con 0x7fe1c80cd900 2024-02-21T23:26:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.876+0000 7fe1c559a640 1 -- 172.21.15.133:0/1322701081 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0034610 con 0x7fe1c80cd900 2024-02-21T23:26:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.876+0000 7fe1cccc8640 1 -- 172.21.15.133:0/1322701081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80cd900 msgr2=0x7fe1c80cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.876+0000 7fe1cccc8640 1 --2- 172.21.15.133:0/1322701081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80cd900 0x7fe1c80cdd70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009e50 tx=0x7fe1c002bff0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.877+0000 7fe1cccc8640 1 -- 172.21.15.133:0/1322701081 shutdown_connections 2024-02-21T23:26:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.877+0000 7fe1cccc8640 1 --2- 172.21.15.133:0/1322701081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80cd900 0x7fe1c80cdd70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.877+0000 7fe1cccc8640 1 --2- 172.21.15.133:0/1322701081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 0x7fe1c80c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.877+0000 7fe1cccc8640 1 -- 172.21.15.133:0/1322701081 >> 172.21.15.133:0/1322701081 conn(0x7fe1c806f7f0 msgr2=0x7fe1c8061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.877+0000 7fe1cccc8640 1 -- 172.21.15.133:0/1322701081 shutdown_connections 2024-02-21T23:26:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.877+0000 7fe1cccc8640 1 -- 172.21.15.133:0/1322701081 wait complete. 2024-02-21T23:26:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.878+0000 7fe1cccc8640 1 Processor -- start 2024-02-21T23:26:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.878+0000 7fe1cccc8640 1 -- start start 2024-02-21T23:26:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1cccc8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80c4600 0x7fe1c814f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1cccc8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c80cd900 0x7fe1c81f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1cccc8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 0x7fe1c81f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1cccc8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c815af60 con 0x7fe1c80cd900 2024-02-21T23:26:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1cccc8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c815ade0 con 0x7fe1c80c4600 2024-02-21T23:26:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1cccc8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80c4a20 con 0x7fe1c8154120 2024-02-21T23:26:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1c6d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 0x7fe1c81f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1c659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80c4600 0x7fe1c814f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1c5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c80cd900 0x7fe1c81f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:00.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1c659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80c4600 0x7fe1c814f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:42946/0 (socket says 172.21.15.133:42946) 2024-02-21T23:26:00.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.879+0000 7fe1c659c640 1 -- 172.21.15.133:0/2073423596 learned_addr learned my addr 172.21.15.133:0/2073423596 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:00.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.883+0000 7fe1c6d9d640 1 -- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80c4600 msgr2=0x7fe1c814f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.883+0000 7fe1c6d9d640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80c4600 0x7fe1c814f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.883+0000 7fe1c6d9d640 1 -- 172.21.15.133:0/2073423596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c80cd900 msgr2=0x7fe1c81f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:00.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.883+0000 7fe1c6d9d640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c80cd900 0x7fe1c81f0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:00.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.883+0000 7fe1c6d9d640 1 -- 172.21.15.133:0/2073423596 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c81ef950 con 0x7fe1c8154120 2024-02-21T23:26:00.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.883+0000 7fe1c659c640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80c4600 0x7fe1c814f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:26:00.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.883+0000 7fe1c5d9b640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c80cd900 0x7fe1c81f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:26:00.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.885+0000 7fe1c6d9d640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 0x7fe1c81f2ec0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0032000 tx=0x7fe1c0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:00.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.890+0000 7fe1b6ffd640 1 -- 172.21.15.133:0/2073423596 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0043020 con 0x7fe1c8154120 2024-02-21T23:26:00.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.890+0000 7fe1b6ffd640 1 -- 172.21.15.133:0/2073423596 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c003e450 con 0x7fe1c8154120 2024-02-21T23:26:00.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.890+0000 7fe1b6ffd640 1 -- 172.21.15.133:0/2073423596 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c003e840 con 0x7fe1c8154120 2024-02-21T23:26:00.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.890+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c81f3620 con 0x7fe1c8154120 2024-02-21T23:26:00.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.890+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c81f3be0 con 0x7fe1c8154120 2024-02-21T23:26:00.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:00.891+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c806a130 con 0x7fe1c8154120 2024-02-21T23:26:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.186+0000 7fe1b6ffd640 1 -- 172.21.15.133:0/2073423596 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1c0046020 con 0x7fe1c8154120 2024-02-21T23:26:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.187+0000 7fe1b6ffd640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1a806b4d0 0x7fe1a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.187+0000 7fe1c659c640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1a806b4d0 0x7fe1a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.187+0000 7fe1c659c640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1a806b4d0 0x7fe1a806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009f50 tx=0x7fe1bc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.189+0000 7fe1b6ffd640 1 -- 172.21.15.133:0/2073423596 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fe1c0079330 con 0x7fe1c8154120 2024-02-21T23:26:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.200+0000 7fe1b6ffd640 1 -- 172.21.15.133:0/2073423596 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fe1c004d030 con 0x7fe1c8154120 2024-02-21T23:26:01.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.371+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1c8071ac0 con 0x7fe1c8154120 2024-02-21T23:26:01.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.379+0000 7fe1b6ffd640 1 -- 172.21.15.133:0/2073423596 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1c007b290 con 0x7fe1c8154120 2024-02-21T23:26:01.365 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:26:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1a806b4d0 msgr2=0x7fe1a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1a806b4d0 0x7fe1a806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009f50 tx=0x7fe1bc009190 comp rx=0 tx=0).stop 2024-02-21T23:26:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 msgr2=0x7fe1c81f2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 0x7fe1c81f2ec0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0032000 tx=0x7fe1c0004a40 comp rx=0 tx=0).stop 2024-02-21T23:26:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 shutdown_connections 2024-02-21T23:26:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe1a806b4d0 0x7fe1a806d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c8154120 0x7fe1c81f2ec0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c80cd900 0x7fe1c81f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 --2- 172.21.15.133:0/2073423596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c80c4600 0x7fe1c814f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 >> 172.21.15.133:0/2073423596 conn(0x7fe1c806f7f0 msgr2=0x7fe1c8062130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 shutdown_connections 2024-02-21T23:26:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.381+0000 7fe1cccc8640 1 -- 172.21.15.133:0/2073423596 wait complete. 2024-02-21T23:26:01.378 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-21T23:26:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.495+0000 7f3d574de640 1 Processor -- start 2024-02-21T23:26:01.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.496+0000 7f3d574de640 1 -- start start 2024-02-21T23:26:01.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.496+0000 7f3d574de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c5fd0 0x7f3d5006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:01.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.496+0000 7f3d574de640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d50070b40 con 0x7f3d501587b0 2024-02-21T23:26:01.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.496+0000 7f3d574de640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d500c7af0 con 0x7f3d500c63b0 2024-02-21T23:26:01.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.496+0000 7f3d574de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d500cd460 con 0x7f3d500c5fd0 2024-02-21T23:26:01.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.497+0000 7f3d54a79640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f3d500c63b0 0x7f3d501580a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:41448/0 (socket says 172.21.15.133:41448) 2024-02-21T23:26:01.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.497+0000 7f3d54a79640 1 -- 172.21.15.133:0/478308455 learned_addr learned my addr 172.21.15.133:0/478308455 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:26:01.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.497+0000 7f3d5527a640 1 --2- 172.21.15.133:0/478308455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c5fd0 0x7f3d5006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:01.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.500+0000 7f3d47fff640 1 -- 172.21.15.133:0/478308455 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10032599 0 0) 0x7f3d50070b40 con 0x7f3d501587b0 2024-02-21T23:26:01.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.500+0000 7f3d47fff640 1 -- 172.21.15.133:0/478308455 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d3c003520 con 0x7f3d501587b0 2024-02-21T23:26:01.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.500+0000 7f3d47fff640 1 -- 172.21.15.133:0/478308455 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817841503 0 0) 0x7f3d40001110 con 0x7f3d500c63b0 2024-02-21T23:26:01.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.500+0000 7f3d47fff640 1 -- 172.21.15.133:0/478308455 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d50070b40 con 0x7f3d500c63b0 2024-02-21T23:26:01.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.500+0000 7f3d5527a640 1 -- 172.21.15.133:0/478308455 >> v1:172.21.15.142:6789/0 conn(0x7f3d500c63b0 legacy=0x7f3d501580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:01.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.500+0000 7f3d5527a640 1 -- 172.21.15.133:0/478308455 >> v1:172.21.15.133:6789/0 conn(0x7f3d501587b0 legacy=0x7f3d50151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:01.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.500+0000 7f3d5527a640 1 -- 172.21.15.133:0/478308455 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d50070860 con 0x7f3d500c5fd0 2024-02-21T23:26:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.502+0000 7f3d5527a640 1 --2- 172.21.15.133:0/478308455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c5fd0 0x7f3d5006b7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009dc0 tx=0x7f3d4c02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc17c9e88b58ca0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:01.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.506+0000 7f3d47fff640 1 -- 172.21.15.133:0/478308455 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c039070 con 0x7f3d500c5fd0 2024-02-21T23:26:01.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.506+0000 7f3d47fff640 1 -- 172.21.15.133:0/478308455 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c02cd50 con 0x7f3d500c5fd0 2024-02-21T23:26:01.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.506+0000 7f3d47fff640 1 -- 172.21.15.133:0/478308455 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c034660 con 0x7f3d500c5fd0 2024-02-21T23:26:01.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.507+0000 7f3d574de640 1 -- 172.21.15.133:0/478308455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c5fd0 msgr2=0x7f3d5006b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:01.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.507+0000 7f3d574de640 1 --2- 172.21.15.133:0/478308455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c5fd0 0x7f3d5006b7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009dc0 tx=0x7f3d4c02bfa0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.507+0000 7f3d574de640 1 -- 172.21.15.133:0/478308455 shutdown_connections 2024-02-21T23:26:01.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.507+0000 7f3d574de640 1 --2- 172.21.15.133:0/478308455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c5fd0 0x7f3d5006b7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.507+0000 7f3d574de640 1 -- 172.21.15.133:0/478308455 >> 172.21.15.133:0/478308455 conn(0x7f3d50061b50 msgr2=0x7f3d50072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:01.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.507+0000 7f3d574de640 1 -- 172.21.15.133:0/478308455 shutdown_connections 2024-02-21T23:26:01.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.507+0000 7f3d574de640 1 -- 172.21.15.133:0/478308455 wait complete. 2024-02-21T23:26:01.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.508+0000 7f3d574de640 1 Processor -- start 2024-02-21T23:26:01.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.508+0000 7f3d574de640 1 -- start start 2024-02-21T23:26:01.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.509+0000 7f3d574de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d500c5fd0 0x7f3d500c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:01.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.509+0000 7f3d574de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c63b0 0x7f3d500cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:01.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.509+0000 7f3d574de640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d501587b0 0x7f3d501f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:01.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.509+0000 7f3d574de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d500cd0f0 con 0x7f3d500c5fd0 2024-02-21T23:26:01.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.509+0000 7f3d574de640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d500cce10 con 0x7f3d501587b0 2024-02-21T23:26:01.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.509+0000 7f3d574de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5006bce0 con 0x7f3d500c63b0 2024-02-21T23:26:01.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.509+0000 7f3d5527a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d500c5fd0 0x7f3d500c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:01.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.509+0000 7f3d55a7b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d501587b0 0x7f3d501f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:01.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.509+0000 7f3d54a79640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c63b0 0x7f3d500cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:01.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.510+0000 7f3d5527a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d500c5fd0 0x7f3d500c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46032/0 (socket says 172.21.15.133:46032) 2024-02-21T23:26:01.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.510+0000 7f3d55a7b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d501587b0 0x7f3d501f06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48032/0 (socket says 172.21.15.133:48032) 2024-02-21T23:26:01.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.510+0000 7f3d5527a640 1 -- 172.21.15.133:0/1435518999 learned_addr learned my addr 172.21.15.133:0/1435518999 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:01.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.513+0000 7f3d5527a640 1 -- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c63b0 msgr2=0x7f3d500cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:01.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.513+0000 7f3d5527a640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c63b0 0x7f3d500cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.513+0000 7f3d5527a640 1 -- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d501587b0 msgr2=0x7f3d501f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:01.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.513+0000 7f3d5527a640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d501587b0 0x7f3d501f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.513+0000 7f3d5527a640 1 -- 172.21.15.133:0/1435518999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5014ada0 con 0x7f3d500c5fd0 2024-02-21T23:26:01.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.515+0000 7f3d5527a640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d500c5fd0 0x7f3d500c92a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009ee0 tx=0x7f3d4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:01.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.517+0000 7f3d45ffb640 1 -- 172.21.15.133:0/1435518999 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c044020 con 0x7f3d500c5fd0 2024-02-21T23:26:01.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.517+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d50149230 con 0x7f3d500c5fd0 2024-02-21T23:26:01.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.518+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d501497f0 con 0x7f3d500c5fd0 2024-02-21T23:26:01.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.518+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5006a130 con 0x7f3d500c5fd0 2024-02-21T23:26:01.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.520+0000 7f3d45ffb640 1 -- 172.21.15.133:0/1435518999 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c02cd50 con 0x7f3d500c5fd0 2024-02-21T23:26:01.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.520+0000 7f3d45ffb640 1 -- 172.21.15.133:0/1435518999 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c039040 con 0x7f3d500c5fd0 2024-02-21T23:26:01.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.766+0000 7f3d45ffb640 1 -- 172.21.15.133:0/1435518999 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d4c047020 con 0x7f3d500c5fd0 2024-02-21T23:26:01.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.767+0000 7f3d45ffb640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3d3406b4d0 0x7f3d3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:01.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.767+0000 7f3d54a79640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3d3406b4d0 0x7f3d3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:01.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.767+0000 7f3d54a79640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3d3406b4d0 0x7f3d3406d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d5014b2f0 tx=0x7f3d40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:01.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.784+0000 7f3d45ffb640 1 -- 172.21.15.133:0/1435518999 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f3d4c079e90 con 0x7f3d500c5fd0 2024-02-21T23:26:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.784+0000 7f3d45ffb640 1 -- 172.21.15.133:0/1435518999 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f3d4c04e020 con 0x7f3d500c5fd0 2024-02-21T23:26:01.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.963+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d50151210 con 0x7f3d500c5fd0 2024-02-21T23:26:01.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.968+0000 7f3d45ffb640 1 -- 172.21.15.133:0/1435518999 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d4c07bdf0 con 0x7f3d500c5fd0 2024-02-21T23:26:01.953 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-02-21T23:26:01.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3d3406b4d0 msgr2=0x7f3d3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:01.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3d3406b4d0 0x7f3d3406d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d5014b2f0 tx=0x7f3d40009040 comp rx=0 tx=0).stop 2024-02-21T23:26:01.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d500c5fd0 msgr2=0x7f3d500c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:01.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d500c5fd0 0x7f3d500c92a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009ee0 tx=0x7f3d4c004a40 comp rx=0 tx=0).stop 2024-02-21T23:26:01.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 shutdown_connections 2024-02-21T23:26:01.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3d3406b4d0 0x7f3d3406d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d501587b0 0x7f3d501f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d500c63b0 0x7f3d500cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 --2- 172.21.15.133:0/1435518999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d500c5fd0 0x7f3d500c92a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:01.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 >> 172.21.15.133:0/1435518999 conn(0x7f3d50061b50 msgr2=0x7f3d5014f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:01.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 shutdown_connections 2024-02-21T23:26:01.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:01.970+0000 7f3d574de640 1 -- 172.21.15.133:0/1435518999 wait complete. 2024-02-21T23:26:01.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-21T23:26:02.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.085+0000 7f05bdb99640 1 Processor -- start 2024-02-21T23:26:02.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.085+0000 7f05bdb99640 1 -- start start 2024-02-21T23:26:02.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.085+0000 7f05bdb99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80cd900 0x7f05b80c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.086+0000 7f05bdb99640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f05b80c9180 0x7f05b81588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.086+0000 7f05bdb99640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b80ce170 con 0x7f05b80cdce0 2024-02-21T23:26:02.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.086+0000 7f05bdb99640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b8158460 con 0x7f05b80c9180 2024-02-21T23:26:02.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.086+0000 7f05bdb99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b815bc10 con 0x7f05b80cd900 2024-02-21T23:26:02.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.086+0000 7f05b7fff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f05b80cdce0 0x7f05b815b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:41462/0 (socket says 172.21.15.133:41462) 2024-02-21T23:26:02.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.086+0000 7f05b7fff640 1 -- 172.21.15.133:0/674225246 learned_addr learned my addr 172.21.15.133:0/674225246 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:26:02.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.086+0000 7f05b77fe640 1 --2- 172.21.15.133:0/674225246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80cd900 0x7f05b80c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.086+0000 7f05b6ffd640 1 --2- 172.21.15.133:0/674225246 >> v2:172.21.15.133:3300/0 conn(0x7f05b80c9180 0x7f05b81588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.089+0000 7f05b67fc640 1 -- 172.21.15.133:0/674225246 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479321697 0 0) 0x7f05b80ce170 con 0x7f05b80cdce0 2024-02-21T23:26:02.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.089+0000 7f05b67fc640 1 -- 172.21.15.133:0/674225246 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a0003520 con 0x7f05b80cdce0 2024-02-21T23:26:02.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.089+0000 7f05b6ffd640 1 -- 172.21.15.133:0/674225246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80cd900 msgr2=0x7f05b80c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:02.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.089+0000 7f05b6ffd640 1 --2- 172.21.15.133:0/674225246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80cd900 0x7f05b80c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.089+0000 7f05b6ffd640 1 -- 172.21.15.133:0/674225246 >> v1:172.21.15.142:6789/0 conn(0x7f05b80cdce0 legacy=0x7f05b815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.089+0000 7f05b6ffd640 1 -- 172.21.15.133:0/674225246 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b815bef0 con 0x7f05b80c9180 2024-02-21T23:26:02.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.091+0000 7f05b6ffd640 1 --2- 172.21.15.133:0/674225246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80c9180 0x7f05b81588f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0097a0 tx=0x7f05ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86855b32a03989a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:02.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.093+0000 7f05b67fc640 1 -- 172.21.15.133:0/674225246 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0315a0 con 0x7f05b80c9180 2024-02-21T23:26:02.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.094+0000 7f05b67fc640 1 -- 172.21.15.133:0/674225246 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac037070 con 0x7f05b80c9180 2024-02-21T23:26:02.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.094+0000 7f05b67fc640 1 -- 172.21.15.133:0/674225246 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac031940 con 0x7f05b80c9180 2024-02-21T23:26:02.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.095+0000 7f05bdb99640 1 -- 172.21.15.133:0/674225246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80c9180 msgr2=0x7f05b81588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.095+0000 7f05bdb99640 1 --2- 172.21.15.133:0/674225246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80c9180 0x7f05b81588f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0097a0 tx=0x7f05ac02b840 comp rx=0 tx=0).stop 2024-02-21T23:26:02.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.095+0000 7f05bdb99640 1 -- 172.21.15.133:0/674225246 shutdown_connections 2024-02-21T23:26:02.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.095+0000 7f05bdb99640 1 --2- 172.21.15.133:0/674225246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80c9180 0x7f05b81588f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.095+0000 7f05bdb99640 1 --2- 172.21.15.133:0/674225246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80cd900 0x7f05b80c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.095+0000 7f05bdb99640 1 -- 172.21.15.133:0/674225246 >> 172.21.15.133:0/674225246 conn(0x7f05b806f7f0 msgr2=0x7f05b8061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:02.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.095+0000 7f05bdb99640 1 -- 172.21.15.133:0/674225246 shutdown_connections 2024-02-21T23:26:02.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.095+0000 7f05bdb99640 1 -- 172.21.15.133:0/674225246 wait complete. 2024-02-21T23:26:02.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.096+0000 7f05bdb99640 1 Processor -- start 2024-02-21T23:26:02.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.096+0000 7f05bdb99640 1 -- start start 2024-02-21T23:26:02.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.097+0000 7f05bdb99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80c9180 0x7f05b806dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.097+0000 7f05bdb99640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b80cd900 0x7f05b8071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.097+0000 7f05bdb99640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80cdce0 0x7f05b8148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.097+0000 7f05bdb99640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81580f0 con 0x7f05b80cdce0 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.097+0000 7f05bdb99640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8157e10 con 0x7f05b80cd900 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.097+0000 7f05bdb99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b80b3a10 con 0x7f05b80c9180 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.098+0000 7f05b6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b80cd900 0x7f05b8071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.098+0000 7f05b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80cdce0 0x7f05b8148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.098+0000 7f05b77fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80c9180 0x7f05b806dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.098+0000 7f05b6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b80cd900 0x7f05b8071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48040/0 (socket says 172.21.15.133:48040) 2024-02-21T23:26:02.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.098+0000 7f05b6ffd640 1 -- 172.21.15.133:0/4132181580 learned_addr learned my addr 172.21.15.133:0/4132181580 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:02.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.101+0000 7f05b7fff640 1 -- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80c9180 msgr2=0x7f05b806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.101+0000 7f05b7fff640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80c9180 0x7f05b806dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.101+0000 7f05b7fff640 1 -- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b80cd900 msgr2=0x7f05b8071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.101+0000 7f05b7fff640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b80cd900 0x7f05b8071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.101+0000 7f05b7fff640 1 -- 172.21.15.133:0/4132181580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b81f00c0 con 0x7f05b80cdce0 2024-02-21T23:26:02.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.104+0000 7f05b7fff640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80cdce0 0x7f05b8148250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05b000ca80 tx=0x7f05b000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:02.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.107+0000 7f059ffff640 1 -- 172.21.15.133:0/4132181580 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0017070 con 0x7f05b80cdce0 2024-02-21T23:26:02.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.107+0000 7f059ffff640 1 -- 172.21.15.133:0/4132181580 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b000ac30 con 0x7f05b80cdce0 2024-02-21T23:26:02.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.107+0000 7f059ffff640 1 -- 172.21.15.133:0/4132181580 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0012600 con 0x7f05b80cdce0 2024-02-21T23:26:02.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.107+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b81f03a0 con 0x7f05b80cdce0 2024-02-21T23:26:02.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.107+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b81f0960 con 0x7f05b80cdce0 2024-02-21T23:26:02.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.108+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b8062500 con 0x7f05b80cdce0 2024-02-21T23:26:02.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.351+0000 7f059ffff640 1 -- 172.21.15.133:0/4132181580 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05b001a070 con 0x7f05b80cdce0 2024-02-21T23:26:02.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.351+0000 7f059ffff640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05a406b4d0 0x7f05a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.352+0000 7f05b77fe640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05a406b4d0 0x7f05a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.352+0000 7f05b77fe640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05a406b4d0 0x7f05a406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009aa0 tx=0x7f05a8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:02.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.366+0000 7f059ffff640 1 -- 172.21.15.133:0/4132181580 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f05b0056c70 con 0x7f05b80cdce0 2024-02-21T23:26:02.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.366+0000 7f059ffff640 1 -- 172.21.15.133:0/4132181580 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f05b8062500 con 0x7f05b80cdce0 2024-02-21T23:26:02.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.537+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f05b806a130 con 0x7f05b80cdce0 2024-02-21T23:26:02.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.543+0000 7f059ffff640 1 -- 172.21.15.133:0/4132181580 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f05b000ae20 con 0x7f05b80cdce0 2024-02-21T23:26:02.529 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:26:02.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05a406b4d0 msgr2=0x7f05a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05a406b4d0 0x7f05a406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009aa0 tx=0x7f05a8009190 comp rx=0 tx=0).stop 2024-02-21T23:26:02.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80cdce0 msgr2=0x7f05b8148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80cdce0 0x7f05b8148250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05b000ca80 tx=0x7f05b000cf40 comp rx=0 tx=0).stop 2024-02-21T23:26:02.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 shutdown_connections 2024-02-21T23:26:02.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05a406b4d0 0x7f05a406d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b80cdce0 0x7f05b8148250 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b80cd900 0x7f05b8071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 --2- 172.21.15.133:0/4132181580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05b80c9180 0x7f05b806dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 >> 172.21.15.133:0/4132181580 conn(0x7f05b806f7f0 msgr2=0x7f05b80cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:02.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 shutdown_connections 2024-02-21T23:26:02.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.545+0000 7f05bdb99640 1 -- 172.21.15.133:0/4132181580 wait complete. 2024-02-21T23:26:02.542 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:26:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.657+0000 7fa93f2de640 1 Processor -- start 2024-02-21T23:26:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.657+0000 7fa93f2de640 1 -- start start 2024-02-21T23:26:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.658+0000 7fa93f2de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa938150f80 0x7fa93815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.658+0000 7fa93f2de640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa93815b020 0x7fa9380cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.658+0000 7fa93f2de640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9380c2d70 con 0x7fa9380cbb30 2024-02-21T23:26:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.658+0000 7fa93f2de640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9380c2b40 con 0x7fa93815b020 2024-02-21T23:26:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.658+0000 7fa93f2de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380c3050 con 0x7fa938150f80 2024-02-21T23:26:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.659+0000 7fa93c879640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa93815b020 0x7fa9380cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.659+0000 7fa93d07a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa938150f80 0x7fa93815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.659+0000 7fa93c879640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa93815b020 0x7fa9380cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46054/0 (socket says 172.21.15.133:46054) 2024-02-21T23:26:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.659+0000 7fa93c879640 1 -- 172.21.15.133:0/2645573375 learned_addr learned my addr 172.21.15.133:0/2645573375 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.661+0000 7fa93d07a640 1 -- 172.21.15.133:0/2645573375 >> v2:172.21.15.133:3300/0 conn(0x7fa93815b020 msgr2=0x7fa9380cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.661+0000 7fa93d07a640 1 --2- 172.21.15.133:0/2645573375 >> v2:172.21.15.133:3300/0 conn(0x7fa93815b020 0x7fa9380cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.661+0000 7fa93d07a640 1 -- 172.21.15.133:0/2645573375 >> v1:172.21.15.133:6789/0 conn(0x7fa9380cbb30 legacy=0x7fa9380c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.661+0000 7fa93d07a640 1 -- 172.21.15.133:0/2645573375 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9380c3330 con 0x7fa938150f80 2024-02-21T23:26:02.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.663+0000 7fa93d07a640 1 --2- 172.21.15.133:0/2645573375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa938150f80 0x7fa93815aae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa934009d90 tx=0x7fa93402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a70b771d298a09 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa92ffff640 1 -- 172.21.15.133:0/2645573375 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934039070 con 0x7fa938150f80 2024-02-21T23:26:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa92ffff640 1 -- 172.21.15.133:0/2645573375 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93402cce0 con 0x7fa938150f80 2024-02-21T23:26:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa92ffff640 1 -- 172.21.15.133:0/2645573375 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934034660 con 0x7fa938150f80 2024-02-21T23:26:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa93f2de640 1 -- 172.21.15.133:0/2645573375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa938150f80 msgr2=0x7fa93815aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa93f2de640 1 --2- 172.21.15.133:0/2645573375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa938150f80 0x7fa93815aae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa934009d90 tx=0x7fa93402bf30 comp rx=0 tx=0).stop 2024-02-21T23:26:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa93f2de640 1 -- 172.21.15.133:0/2645573375 shutdown_connections 2024-02-21T23:26:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa93f2de640 1 --2- 172.21.15.133:0/2645573375 >> v2:172.21.15.133:3300/0 conn(0x7fa93815b020 0x7fa9380cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa93f2de640 1 --2- 172.21.15.133:0/2645573375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa938150f80 0x7fa93815aae0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa93f2de640 1 -- 172.21.15.133:0/2645573375 >> 172.21.15.133:0/2645573375 conn(0x7fa93806aec0 msgr2=0x7fa93806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa93f2de640 1 -- 172.21.15.133:0/2645573375 shutdown_connections 2024-02-21T23:26:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.668+0000 7fa93f2de640 1 -- 172.21.15.133:0/2645573375 wait complete. 2024-02-21T23:26:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.670+0000 7fa93f2de640 1 Processor -- start 2024-02-21T23:26:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.670+0000 7fa93f2de640 1 -- start start 2024-02-21T23:26:02.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.670+0000 7fa93f2de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380cbb30 0x7fa9381e06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.670+0000 7fa93f2de640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938150f80 0x7fa9381e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.670+0000 7fa93f2de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa93815b020 0x7fa9381e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.670+0000 7fa93f2de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380c5990 con 0x7fa9380cbb30 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.670+0000 7fa93f2de640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93815a5c0 con 0x7fa938150f80 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.670+0000 7fa93f2de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380c5b10 con 0x7fa93815b020 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.671+0000 7fa93d07a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380cbb30 0x7fa9381e06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.671+0000 7fa93d87b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa93815b020 0x7fa9381e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.671+0000 7fa93c879640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938150f80 0x7fa9381e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.671+0000 7fa93d07a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380cbb30 0x7fa9381e06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46056/0 (socket says 172.21.15.133:46056) 2024-02-21T23:26:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.671+0000 7fa93d07a640 1 -- 172.21.15.133:0/205767419 learned_addr learned my addr 172.21.15.133:0/205767419 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:02.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.674+0000 7fa93d07a640 1 -- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa93815b020 msgr2=0x7fa9381e14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.674+0000 7fa93d07a640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa93815b020 0x7fa9381e14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.674+0000 7fa93d07a640 1 -- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938150f80 msgr2=0x7fa9381e0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:02.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.674+0000 7fa93d07a640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938150f80 0x7fa9381e0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:02.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.674+0000 7fa93d07a640 1 -- 172.21.15.133:0/205767419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9381ee9b0 con 0x7fa9380cbb30 2024-02-21T23:26:02.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.677+0000 7fa93d07a640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380cbb30 0x7fa9381e06b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa934037040 tx=0x7fa934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:02.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.680+0000 7fa92dffb640 1 -- 172.21.15.133:0/205767419 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934039050 con 0x7fa9380cbb30 2024-02-21T23:26:02.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.680+0000 7fa92dffb640 1 -- 172.21.15.133:0/205767419 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa934045070 con 0x7fa9380cbb30 2024-02-21T23:26:02.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.680+0000 7fa92dffb640 1 -- 172.21.15.133:0/205767419 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93403e850 con 0x7fa9380cbb30 2024-02-21T23:26:02.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.680+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa938149e70 con 0x7fa9380cbb30 2024-02-21T23:26:02.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.681+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93814a430 con 0x7fa9380cbb30 2024-02-21T23:26:02.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.681+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93806a130 con 0x7fa9380cbb30 2024-02-21T23:26:02.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.926+0000 7fa92dffb640 1 -- 172.21.15.133:0/205767419 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa93403e9f0 con 0x7fa9380cbb30 2024-02-21T23:26:02.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.926+0000 7fa92dffb640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa91c06b550 0x7fa91c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:02.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.927+0000 7fa93c879640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa91c06b550 0x7fa91c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:02.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.927+0000 7fa93c879640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa91c06b550 0x7fa91c06da10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9380ca6a0 tx=0x7fa928009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.956+0000 7fa92dffb640 1 -- 172.21.15.133:0/205767419 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fa93407a1a0 con 0x7fa9380cbb30 2024-02-21T23:26:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:02.957+0000 7fa92dffb640 1 -- 172.21.15.133:0/205767419 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fa93407c100 con 0x7fa9380cbb30 2024-02-21T23:26:03.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.198+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa93806b560 con 0x7fa9380cbb30 2024-02-21T23:26:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.211+0000 7fa92dffb640 1 -- 172.21.15.133:0/205767419 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6535 (secure 0 0 0) 0x7fa9340801a0 con 0x7fa9380cbb30 2024-02-21T23:26:03.197 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:26:03.197 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1319340142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1319340142},{"type":"v1","addr":"172.21.15.133:6837","nonce":1319340142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1740931674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1740931674},{"type":"v1","addr":"172.21.15.142:6837","nonce":1740931674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3712864993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3712864993},{"type":"v1","addr":"172.21.15.142:6839","nonce":3712864993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2928905924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2928905924},{"type":"v1","addr":"172.21.15.133:6833","nonce":2928905924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3461010406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3461010406},{"type":"v1","addr":"172.21.15.133:6835","nonce":3461010406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-21T23:25:16.775942+0000","modified":"2024-02-21T23:25:49.141553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6835/1527124199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1527124199},{"type":"v1","addr":"172.21.15.142:6835","nonce":1527124199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-21T23:26:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2024-02-21T23:26:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa91c06b550 msgr2=0x7fa91c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa91c06b550 0x7fa91c06da10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9380ca6a0 tx=0x7fa928009240 comp rx=0 tx=0).stop 2024-02-21T23:26:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380cbb30 msgr2=0x7fa9381e06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380cbb30 0x7fa9381e06b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa934037040 tx=0x7fa934004a40 comp rx=0 tx=0).stop 2024-02-21T23:26:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 shutdown_connections 2024-02-21T23:26:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa91c06b550 0x7fa91c06da10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa93815b020 0x7fa9381e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938150f80 0x7fa9381e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 --2- 172.21.15.133:0/205767419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380cbb30 0x7fa9381e06b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 >> 172.21.15.133:0/205767419 conn(0x7fa93806aec0 msgr2=0x7fa9380c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 shutdown_connections 2024-02-21T23:26:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.213+0000 7fa93f2de640 1 -- 172.21.15.133:0/205767419 wait complete. 2024-02-21T23:26:03.210 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:26:03.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.325+0000 7f1bb2a36640 1 Processor -- start 2024-02-21T23:26:03.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.325+0000 7f1bb2a36640 1 -- start start 2024-02-21T23:26:03.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.326+0000 7f1bb2a36640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1bac0c4fc0 0x7f1bac071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.326+0000 7f1bb2a36640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bac158230 con 0x7f1bac0c53a0 2024-02-21T23:26:03.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.326+0000 7f1bb2a36640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bac158510 con 0x7f1bac15b7b0 2024-02-21T23:26:03.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.326+0000 7f1bb2a36640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac158eb0 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.327+0000 7f1bab7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1bac0c53a0 0x7f1bac06f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45160/0 (socket says 172.21.15.133:45160) 2024-02-21T23:26:03.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.327+0000 7f1bab7fe640 1 -- 172.21.15.133:0/1559288621 learned_addr learned my addr 172.21.15.133:0/1559288621 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:26:03.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.327+0000 7f1babfff640 1 --2- 172.21.15.133:0/1559288621 >> v2:172.21.15.142:3300/0 conn(0x7f1bac0c4fc0 0x7f1bac071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.329+0000 7f1baaffd640 1 -- 172.21.15.133:0/1559288621 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243617143 0 0) 0x7f1bac158230 con 0x7f1bac0c53a0 2024-02-21T23:26:03.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.329+0000 7f1baaffd640 1 -- 172.21.15.133:0/1559288621 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b94003520 con 0x7f1bac0c53a0 2024-02-21T23:26:03.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.330+0000 7f1babfff640 1 -- 172.21.15.133:0/1559288621 >> v1:172.21.15.142:6789/0 conn(0x7f1bac15b7b0 legacy=0x7f1bac15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.330+0000 7f1babfff640 1 -- 172.21.15.133:0/1559288621 >> v1:172.21.15.133:6789/0 conn(0x7f1bac0c53a0 legacy=0x7f1bac06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.330+0000 7f1babfff640 1 -- 172.21.15.133:0/1559288621 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac1587f0 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.334+0000 7f1babfff640 1 --2- 172.21.15.133:0/1559288621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 0x7f1bac071b60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0097a0 tx=0x7f1b9c02b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b792cf0530627195 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:03.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.337+0000 7f1baaffd640 1 -- 172.21.15.133:0/1559288621 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c02f810 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.337+0000 7f1baaffd640 1 -- 172.21.15.133:0/1559288621 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c02f9b0 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.337+0000 7f1baaffd640 1 -- 172.21.15.133:0/1559288621 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c02fd80 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.338+0000 7f1bb2a36640 1 -- 172.21.15.133:0/1559288621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 msgr2=0x7f1bac071b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.338+0000 7f1bb2a36640 1 --2- 172.21.15.133:0/1559288621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 0x7f1bac071b60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0097a0 tx=0x7f1b9c02b9f0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.338+0000 7f1bb2a36640 1 -- 172.21.15.133:0/1559288621 shutdown_connections 2024-02-21T23:26:03.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.338+0000 7f1bb2a36640 1 --2- 172.21.15.133:0/1559288621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 0x7f1bac071b60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.338+0000 7f1bb2a36640 1 -- 172.21.15.133:0/1559288621 >> 172.21.15.133:0/1559288621 conn(0x7f1bac06aec0 msgr2=0x7f1bac06b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:03.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.338+0000 7f1bb2a36640 1 -- 172.21.15.133:0/1559288621 shutdown_connections 2024-02-21T23:26:03.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.338+0000 7f1bb2a36640 1 -- 172.21.15.133:0/1559288621 wait complete. 2024-02-21T23:26:03.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.339+0000 7f1bb2a36640 1 Processor -- start 2024-02-21T23:26:03.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.340+0000 7f1bb2a36640 1 -- start start 2024-02-21T23:26:03.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.340+0000 7f1bb2a36640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 0x7f1bac150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.340+0000 7f1bb2a36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bac0c53a0 0x7f1bac1ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.340+0000 7f1bb2a36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bac15b7b0 0x7f1bac1ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.340+0000 7f1bb2a36640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac06f8d0 con 0x7f1bac0c53a0 2024-02-21T23:26:03.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.340+0000 7f1bb2a36640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac157f20 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.340+0000 7f1bb2a36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac06fa50 con 0x7f1bac15b7b0 2024-02-21T23:26:03.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.341+0000 7f1babfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 0x7f1bac150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.341+0000 7f1babfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 0x7f1bac150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48076/0 (socket says 172.21.15.133:48076) 2024-02-21T23:26:03.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.341+0000 7f1babfff640 1 -- 172.21.15.133:0/2033435940 learned_addr learned my addr 172.21.15.133:0/2033435940 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:03.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.342+0000 7f1bab7fe640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bac0c53a0 0x7f1bac1ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.343+0000 7f1bb0fd3640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bac15b7b0 0x7f1bac1ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.345+0000 7f1babfff640 1 -- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bac15b7b0 msgr2=0x7f1bac1ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.345+0000 7f1babfff640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bac15b7b0 0x7f1bac1ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.345+0000 7f1babfff640 1 -- 172.21.15.133:0/2033435940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bac0c53a0 msgr2=0x7f1bac1ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.345+0000 7f1babfff640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bac0c53a0 0x7f1bac1ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.345+0000 7f1babfff640 1 -- 172.21.15.133:0/2033435940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac1f2a90 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.347+0000 7f1babfff640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 0x7f1bac150870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0098c0 tx=0x7f1b9c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:03.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.352+0000 7f1ba8ff9640 1 -- 172.21.15.133:0/2033435940 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c047070 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.352+0000 7f1ba8ff9640 1 -- 172.21.15.133:0/2033435940 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c0378d0 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.352+0000 7f1ba8ff9640 1 -- 172.21.15.133:0/2033435940 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c037cc0 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.352+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac1f2d30 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.352+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac1f32f0 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.353+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bac06a130 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.379+0000 7f1ba8ff9640 1 -- 172.21.15.133:0/2033435940 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b9c04a070 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.379+0000 7f1ba8ff9640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1b9806b520 0x7f1b9806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.380+0000 7f1bab7fe640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1b9806b520 0x7f1b9806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.380+0000 7f1bab7fe640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1b9806b520 0x7f1b9806d9e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0c9c60 tx=0x7f1ba0007d30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:03.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.383+0000 7f1ba8ff9640 1 -- 172.21.15.133:0/2033435940 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f1b9c078ab0 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.649+0000 7f1ba8ff9640 1 -- 172.21.15.133:0/2033435940 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f1b9c051080 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.822+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bac0717a0 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.867+0000 7f1ba8ff9640 1 -- 172.21.15.133:0/2033435940 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19091 (secure 0 0 0) 0x7f1b9c033030 con 0x7f1bac0c4fc0 2024-02-21T23:26:03.853 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:26:03.853 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:57.029812+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":0}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:26:03.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1b9806b520 msgr2=0x7f1b9806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1b9806b520 0x7f1b9806d9e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0c9c60 tx=0x7f1ba0007d30 comp rx=0 tx=0).stop 2024-02-21T23:26:03.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 msgr2=0x7f1bac150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 0x7f1bac150870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0098c0 tx=0x7f1b9c004730 comp rx=0 tx=0).stop 2024-02-21T23:26:03.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 shutdown_connections 2024-02-21T23:26:03.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f1b9806b520 0x7f1b9806d9e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bac15b7b0 0x7f1bac1ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bac0c53a0 0x7f1bac1ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 --2- 172.21.15.133:0/2033435940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac0c4fc0 0x7f1bac150870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 >> 172.21.15.133:0/2033435940 conn(0x7f1bac06aec0 msgr2=0x7f1bac15bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:03.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 shutdown_connections 2024-02-21T23:26:03.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.869+0000 7f1bb2a36640 1 -- 172.21.15.133:0/2033435940 wait complete. 2024-02-21T23:26:03.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-21T23:25:02.902698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-21T23:25:13.993803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-21T23:25:15.604077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-21T23:25:19.468744+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-21T23:26:03.867 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-21T23:26:03.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.987+0000 7fecccad1640 1 Processor -- start 2024-02-21T23:26:03.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.987+0000 7fecccad1640 1 -- start start 2024-02-21T23:26:03.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.988+0000 7fecccad1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fecc80c73d0 0x7fecc806aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.988+0000 7fecccad1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc806d0a0 0x7fecc815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.988+0000 7fecccad1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc806e660 con 0x7fecc8152270 2024-02-21T23:26:03.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.988+0000 7fecccad1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc815b470 con 0x7fecc80c73d0 2024-02-21T23:26:03.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.988+0000 7fecccad1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc8152650 con 0x7fecc806d0a0 2024-02-21T23:26:03.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.988+0000 7fecbffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fecc80c73d0 0x7fecc806aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.988+0000 7fecc6d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc806d0a0 0x7fecc815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.989+0000 7fecc6d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc806d0a0 0x7fecc815bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:52698/0 (socket says 172.21.15.133:52698) 2024-02-21T23:26:03.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.989+0000 7fecc6d9d640 1 -- 172.21.15.133:0/965108386 learned_addr learned my addr 172.21.15.133:0/965108386 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.991+0000 7fecc6d9d640 1 -- 172.21.15.133:0/965108386 >> v2:172.21.15.133:3300/0 conn(0x7fecc80c73d0 msgr2=0x7fecc806aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.991+0000 7fecc6d9d640 1 --2- 172.21.15.133:0/965108386 >> v2:172.21.15.133:3300/0 conn(0x7fecc80c73d0 0x7fecc806aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.991+0000 7fecc6d9d640 1 -- 172.21.15.133:0/965108386 >> v1:172.21.15.133:6789/0 conn(0x7fecc8152270 legacy=0x7fecc8072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.991+0000 7fecc6d9d640 1 -- 172.21.15.133:0/965108386 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc806e940 con 0x7fecc806d0a0 2024-02-21T23:26:03.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.993+0000 7fecc6d9d640 1 --2- 172.21.15.133:0/965108386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc806d0a0 0x7fecc815bec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009e50 tx=0x7fecc002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f1c29f9279f92a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:03.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.997+0000 7fecc5d9b640 1 -- 172.21.15.133:0/965108386 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc002ca70 con 0x7fecc806d0a0 2024-02-21T23:26:03.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecc5d9b640 1 -- 172.21.15.133:0/965108386 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc002cc10 con 0x7fecc806d0a0 2024-02-21T23:26:03.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecc5d9b640 1 -- 172.21.15.133:0/965108386 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc00343f0 con 0x7fecc806d0a0 2024-02-21T23:26:03.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecccad1640 1 -- 172.21.15.133:0/965108386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc806d0a0 msgr2=0x7fecc815bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecccad1640 1 --2- 172.21.15.133:0/965108386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc806d0a0 0x7fecc815bec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009e50 tx=0x7fecc002bed0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecccad1640 1 -- 172.21.15.133:0/965108386 shutdown_connections 2024-02-21T23:26:03.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecccad1640 1 --2- 172.21.15.133:0/965108386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc806d0a0 0x7fecc815bec0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecccad1640 1 --2- 172.21.15.133:0/965108386 >> v2:172.21.15.133:3300/0 conn(0x7fecc80c73d0 0x7fecc806aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecccad1640 1 -- 172.21.15.133:0/965108386 >> 172.21.15.133:0/965108386 conn(0x7fecc80cc620 msgr2=0x7fecc80c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:03.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecccad1640 1 -- 172.21.15.133:0/965108386 shutdown_connections 2024-02-21T23:26:03.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:03.998+0000 7fecccad1640 1 -- 172.21.15.133:0/965108386 wait complete. 2024-02-21T23:26:03.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.000+0000 7fecccad1640 1 Processor -- start 2024-02-21T23:26:03.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.000+0000 7fecccad1640 1 -- start start 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.000+0000 7fecccad1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecc806d0a0 0x7fecc81548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.000+0000 7fecccad1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecc80c73d0 0x7fecc814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.000+0000 7fecccad1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc8152270 0x7fecc8147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.000+0000 7fecccad1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc815b9a0 con 0x7fecc806d0a0 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.000+0000 7fecccad1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc815b6c0 con 0x7fecc80c73d0 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.000+0000 7fecccad1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc806c410 con 0x7fecc8152270 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.001+0000 7fecbffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecc80c73d0 0x7fecc814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.001+0000 7fecc6d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc8152270 0x7fecc8147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.001+0000 7fecc6d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc8152270 0x7fecc8147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:52704/0 (socket says 172.21.15.133:52704) 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.001+0000 7fecbffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecc80c73d0 0x7fecc814d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48086/0 (socket says 172.21.15.133:48086) 2024-02-21T23:26:03.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.001+0000 7fecc6d9d640 1 -- 172.21.15.133:0/1481425560 learned_addr learned my addr 172.21.15.133:0/1481425560 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:03.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.002+0000 7fecc659c640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecc806d0a0 0x7fecc81548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.005+0000 7fecbffff640 1 -- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc8152270 msgr2=0x7fecc8147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.005+0000 7fecbffff640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc8152270 0x7fecc8147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.005+0000 7fecbffff640 1 -- 172.21.15.133:0/1481425560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecc806d0a0 msgr2=0x7fecc81548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.005+0000 7fecbffff640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecc806d0a0 0x7fecc81548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.005+0000 7fecbffff640 1 -- 172.21.15.133:0/1481425560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc80c22c0 con 0x7fecc80c73d0 2024-02-21T23:26:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.007+0000 7fecbffff640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecc80c73d0 0x7fecc814d0f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e870 tx=0x7fecb000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.013+0000 7fecbeffd640 1 -- 172.21.15.133:0/1481425560 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0017070 con 0x7fecc80c73d0 2024-02-21T23:26:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.013+0000 7fecbeffd640 1 -- 172.21.15.133:0/1481425560 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb000ee60 con 0x7fecc80c73d0 2024-02-21T23:26:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.013+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc80c2b60 con 0x7fecc80c73d0 2024-02-21T23:26:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.013+0000 7fecbeffd640 1 -- 172.21.15.133:0/1481425560 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb000cb60 con 0x7fecc80c73d0 2024-02-21T23:26:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.013+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc8154380 con 0x7fecc80c73d0 2024-02-21T23:26:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.014+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc806a130 con 0x7fecc80c73d0 2024-02-21T23:26:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.268+0000 7fecbeffd640 1 -- 172.21.15.133:0/1481425560 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecb001a070 con 0x7fecc80c73d0 2024-02-21T23:26:04.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.269+0000 7fecbeffd640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feca806b520 0x7feca806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:04.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.269+0000 7fecc659c640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feca806b520 0x7feca806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:04.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.270+0000 7fecc659c640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feca806b520 0x7feca806d9e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fecb8001780 tx=0x7fecb8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.272+0000 7fecbeffd640 1 -- 172.21.15.133:0/1481425560 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fecb0056ff0 con 0x7fecc80c73d0 2024-02-21T23:26:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.283+0000 7fecbeffd640 1 -- 172.21.15.133:0/1481425560 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fecb0058f50 con 0x7fecc80c73d0 2024-02-21T23:26:04.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.460+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecc80c77f0 con 0x7fecc80c73d0 2024-02-21T23:26:04.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.465+0000 7fecbeffd640 1 -- 172.21.15.133:0/1481425560 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fecb005cff0 con 0x7fecc80c73d0 2024-02-21T23:26:04.451 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-02-21T23:26:04.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feca806b520 msgr2=0x7feca806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feca806b520 0x7feca806d9e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fecb8001780 tx=0x7fecb8009040 comp rx=0 tx=0).stop 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecc80c73d0 msgr2=0x7fecc814d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecc80c73d0 0x7fecc814d0f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e870 tx=0x7fecb000ed30 comp rx=0 tx=0).stop 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 shutdown_connections 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feca806b520 0x7feca806d9e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecc8152270 0x7fecc8147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecc80c73d0 0x7fecc814d0f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 --2- 172.21.15.133:0/1481425560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecc806d0a0 0x7fecc81548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 >> 172.21.15.133:0/1481425560 conn(0x7fecc80cc620 msgr2=0x7fecc814fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 shutdown_connections 2024-02-21T23:26:04.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.467+0000 7fecccad1640 1 -- 172.21.15.133:0/1481425560 wait complete. 2024-02-21T23:26:04.465 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-21T23:26:04.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.583+0000 7f27fa836640 1 Processor -- start 2024-02-21T23:26:04.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.584+0000 7f27fa836640 1 -- start start 2024-02-21T23:26:04.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.584+0000 7f27fa836640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f27f4154600 0x7f27f40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:04.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.584+0000 7f27fa836640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f40c6f50 0x7f27f40ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:04.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.584+0000 7f27fa836640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f415ab40 con 0x7f27f40cab80 2024-02-21T23:26:04.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.584+0000 7f27fa836640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f40c74a0 con 0x7f27f4154600 2024-02-21T23:26:04.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.584+0000 7f27fa836640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f40c7780 con 0x7f27f40c6f50 2024-02-21T23:26:04.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.585+0000 7f27f8dd3640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f27f40cab80 0x7f27f40c7c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:41472/0 (socket says 172.21.15.133:41472) 2024-02-21T23:26:04.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.585+0000 7f27f8dd3640 1 -- 172.21.15.133:0/3079724900 learned_addr learned my addr 172.21.15.133:0/3079724900 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:26:04.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.585+0000 7f27f37fe640 1 --2- 172.21.15.133:0/3079724900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f40c6f50 0x7f27f40ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:04.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.585+0000 7f27f3fff640 1 --2- 172.21.15.133:0/3079724900 >> v2:172.21.15.133:3300/0 conn(0x7f27f4154600 0x7f27f40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:04.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.587+0000 7f27f3fff640 1 -- 172.21.15.133:0/3079724900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f40c6f50 msgr2=0x7f27f40ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:04.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.587+0000 7f27f3fff640 1 --2- 172.21.15.133:0/3079724900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f40c6f50 0x7f27f40ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.587+0000 7f27f3fff640 1 -- 172.21.15.133:0/3079724900 >> v1:172.21.15.142:6789/0 conn(0x7f27f40cab80 legacy=0x7f27f40c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:04.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.587+0000 7f27f3fff640 1 -- 172.21.15.133:0/3079724900 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f415ae00 con 0x7f27f4154600 2024-02-21T23:26:04.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.589+0000 7f27f3fff640 1 --2- 172.21.15.133:0/3079724900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f4154600 0x7f27f40cbc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f27e40097a0 tx=0x7f27e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8714d739b62af939 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.593+0000 7f27f2ffd640 1 -- 172.21.15.133:0/3079724900 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4035020 con 0x7f27f4154600 2024-02-21T23:26:04.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.593+0000 7f27f2ffd640 1 -- 172.21.15.133:0/3079724900 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e4031830 con 0x7f27f4154600 2024-02-21T23:26:04.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.593+0000 7f27f2ffd640 1 -- 172.21.15.133:0/3079724900 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4031c00 con 0x7f27f4154600 2024-02-21T23:26:04.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.594+0000 7f27fa836640 1 -- 172.21.15.133:0/3079724900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f4154600 msgr2=0x7f27f40cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.594+0000 7f27fa836640 1 --2- 172.21.15.133:0/3079724900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f4154600 0x7f27f40cbc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f27e40097a0 tx=0x7f27e402b8d0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.594+0000 7f27fa836640 1 -- 172.21.15.133:0/3079724900 shutdown_connections 2024-02-21T23:26:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.594+0000 7f27fa836640 1 --2- 172.21.15.133:0/3079724900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f40c6f50 0x7f27f40ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.594+0000 7f27fa836640 1 --2- 172.21.15.133:0/3079724900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f4154600 0x7f27f40cbc00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.594+0000 7f27fa836640 1 -- 172.21.15.133:0/3079724900 >> 172.21.15.133:0/3079724900 conn(0x7f27f4062000 msgr2=0x7f27f40cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.594+0000 7f27fa836640 1 -- 172.21.15.133:0/3079724900 shutdown_connections 2024-02-21T23:26:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.594+0000 7f27fa836640 1 -- 172.21.15.133:0/3079724900 wait complete. 2024-02-21T23:26:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.595+0000 7f27fa836640 1 Processor -- start 2024-02-21T23:26:04.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.595+0000 7f27fa836640 1 -- start start 2024-02-21T23:26:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27fa836640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40c6f50 0x7f27f41572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27fa836640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27f40cab80 0x7f27f4154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27fa836640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f4154600 0x7f27f4159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27fa836640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f40b5720 con 0x7f27f40c6f50 2024-02-21T23:26:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27fa836640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f40cb860 con 0x7f27f40cab80 2024-02-21T23:26:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27fa836640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f40b58a0 con 0x7f27f4154600 2024-02-21T23:26:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40c6f50 0x7f27f41572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27f37fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27f40cab80 0x7f27f4154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27f8dd3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f4154600 0x7f27f4159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27f37fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27f40cab80 0x7f27f4154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48096/0 (socket says 172.21.15.133:48096) 2024-02-21T23:26:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.596+0000 7f27f37fe640 1 -- 172.21.15.133:0/674621685 learned_addr learned my addr 172.21.15.133:0/674621685 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.600+0000 7f27f3fff640 1 -- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f4154600 msgr2=0x7f27f4159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.600+0000 7f27f3fff640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f4154600 0x7f27f4159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.600+0000 7f27f3fff640 1 -- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27f40cab80 msgr2=0x7f27f4154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:04.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.600+0000 7f27f3fff640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27f40cab80 0x7f27f4154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:04.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.600+0000 7f27f3fff640 1 -- 172.21.15.133:0/674621685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f414e760 con 0x7f27f40c6f50 2024-02-21T23:26:04.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.602+0000 7f27f3fff640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40c6f50 0x7f27f41572f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27e4033000 tx=0x7f27e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:04.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.604+0000 7f27f0ff9640 1 -- 172.21.15.133:0/674621685 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e403a970 con 0x7f27f40c6f50 2024-02-21T23:26:04.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.604+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f4157a30 con 0x7f27f40c6f50 2024-02-21T23:26:04.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.604+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f414ecc0 con 0x7f27f40c6f50 2024-02-21T23:26:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.605+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f406a130 con 0x7f27f40c6f50 2024-02-21T23:26:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.606+0000 7f27f0ff9640 1 -- 172.21.15.133:0/674621685 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e4031830 con 0x7f27f40c6f50 2024-02-21T23:26:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.606+0000 7f27f0ff9640 1 -- 172.21.15.133:0/674621685 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e403ad10 con 0x7f27f40c6f50 2024-02-21T23:26:04.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.851+0000 7f27f0ff9640 1 -- 172.21.15.133:0/674621685 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27e404b060 con 0x7f27f40c6f50 2024-02-21T23:26:04.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.851+0000 7f27f0ff9640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f27e006b4d0 0x7f27e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:04.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.851+0000 7f27f37fe640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f27e006b4d0 0x7f27e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:04.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.852+0000 7f27f37fe640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f27e006b4d0 0x7f27e006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27f4152040 tx=0x7f27e8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.863+0000 7f27f0ff9640 1 -- 172.21.15.133:0/674621685 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f27e4078620 con 0x7f27f40c6f50 2024-02-21T23:26:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:04.863+0000 7f27f0ff9640 1 -- 172.21.15.133:0/674621685 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f27f414e760 con 0x7f27f40c6f50 2024-02-21T23:26:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.064+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27f4154f70 con 0x7f27f40c6f50 2024-02-21T23:26:05.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.070+0000 7f27f0ff9640 1 -- 172.21.15.133:0/674621685 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27e403f070 con 0x7f27f40c6f50 2024-02-21T23:26:05.055 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:26:05.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f27e006b4d0 msgr2=0x7f27e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f27e006b4d0 0x7f27e006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27f4152040 tx=0x7f27e8009190 comp rx=0 tx=0).stop 2024-02-21T23:26:05.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40c6f50 msgr2=0x7f27f41572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40c6f50 0x7f27f41572f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27e4033000 tx=0x7f27e40046c0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 shutdown_connections 2024-02-21T23:26:05.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f27e006b4d0 0x7f27e006d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27f4154600 0x7f27f4159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27f40cab80 0x7f27f4154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 --2- 172.21.15.133:0/674621685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40c6f50 0x7f27f41572f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 >> 172.21.15.133:0/674621685 conn(0x7f27f4062000 msgr2=0x7f27f40cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:05.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 shutdown_connections 2024-02-21T23:26:05.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.072+0000 7f27fa836640 1 -- 172.21.15.133:0/674621685 wait complete. 2024-02-21T23:26:05.069 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-21T23:26:05.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.187+0000 7f5813f84640 1 Processor -- start 2024-02-21T23:26:05.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.188+0000 7f5813f84640 1 -- start start 2024-02-21T23:26:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.189+0000 7f5813f84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c154600 0x7f580c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.189+0000 7f5813f84640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f580c0c6f50 0x7f580c0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.189+0000 7f5813f84640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f580c0cab80 0x7f580c0c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.189+0000 7f5813f84640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c0c74a0 con 0x7f580c0c6f50 2024-02-21T23:26:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.189+0000 7f5813f84640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c0c7780 con 0x7f580c0cab80 2024-02-21T23:26:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.189+0000 7f5813f84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c15ab40 con 0x7f580c154600 2024-02-21T23:26:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.189+0000 7f5811d20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c154600 0x7f580c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.189+0000 7f581151f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f580c0c6f50 0x7f580c0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.189+0000 7f5812521640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f580c0cab80 0x7f580c0c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.190+0000 7f5811d20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c154600 0x7f580c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:52730/0 (socket says 172.21.15.133:52730) 2024-02-21T23:26:05.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.190+0000 7f5812521640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f580c0cab80 0x7f580c0c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48110/0 (socket says 172.21.15.133:48110) 2024-02-21T23:26:05.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.190+0000 7f5811d20640 1 -- 172.21.15.133:0/3063491032 learned_addr learned my addr 172.21.15.133:0/3063491032 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:05.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.192+0000 7f581151f640 1 -- 172.21.15.133:0/3063491032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c154600 msgr2=0x7f580c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:05.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.192+0000 7f581151f640 1 --2- 172.21.15.133:0/3063491032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c154600 0x7f580c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.192+0000 7f581151f640 1 -- 172.21.15.133:0/3063491032 >> v2:172.21.15.142:3300/0 conn(0x7f580c0cab80 msgr2=0x7f580c0c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:05.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.192+0000 7f581151f640 1 --2- 172.21.15.133:0/3063491032 >> v2:172.21.15.142:3300/0 conn(0x7f580c0cab80 0x7f580c0c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.192+0000 7f581151f640 1 -- 172.21.15.133:0/3063491032 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c15ae00 con 0x7f580c0c6f50 2024-02-21T23:26:05.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.194+0000 7f581151f640 1 --2- 172.21.15.133:0/3063491032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0c6f50 0x7f580c0ca640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f580c0c6700 tx=0x7f580802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd21fb920f917f0b server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:05.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5810d1e640 1 -- 172.21.15.133:0/3063491032 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080315a0 con 0x7f580c0c6f50 2024-02-21T23:26:05.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5810d1e640 1 -- 172.21.15.133:0/3063491032 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5808037070 con 0x7f580c0c6f50 2024-02-21T23:26:05.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5810d1e640 1 -- 172.21.15.133:0/3063491032 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808031960 con 0x7f580c0c6f50 2024-02-21T23:26:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5813f84640 1 -- 172.21.15.133:0/3063491032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0c6f50 msgr2=0x7f580c0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5813f84640 1 --2- 172.21.15.133:0/3063491032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0c6f50 0x7f580c0ca640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f580c0c6700 tx=0x7f580802b840 comp rx=0 tx=0).stop 2024-02-21T23:26:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5813f84640 1 -- 172.21.15.133:0/3063491032 shutdown_connections 2024-02-21T23:26:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5813f84640 1 --2- 172.21.15.133:0/3063491032 >> v2:172.21.15.142:3300/0 conn(0x7f580c0cab80 0x7f580c0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5813f84640 1 --2- 172.21.15.133:0/3063491032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0c6f50 0x7f580c0ca640 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5813f84640 1 --2- 172.21.15.133:0/3063491032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c154600 0x7f580c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5813f84640 1 -- 172.21.15.133:0/3063491032 >> 172.21.15.133:0/3063491032 conn(0x7f580c062000 msgr2=0x7f580c0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5813f84640 1 -- 172.21.15.133:0/3063491032 shutdown_connections 2024-02-21T23:26:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.198+0000 7f5813f84640 1 -- 172.21.15.133:0/3063491032 wait complete. 2024-02-21T23:26:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.200+0000 7f5813f84640 1 Processor -- start 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.200+0000 7f5813f84640 1 -- start start 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.200+0000 7f5813f84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c0c6f50 0x7f580c1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.200+0000 7f5813f84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0cab80 0x7f580c158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.200+0000 7f5813f84640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c154600 0x7f580c1f0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.200+0000 7f5813f84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0b5720 con 0x7f580c0cab80 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.200+0000 7f5813f84640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0cb860 con 0x7f580c154600 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.200+0000 7f5813f84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0b58a0 con 0x7f580c0c6f50 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.201+0000 7f581151f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0cab80 0x7f580c158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.201+0000 7f5812521640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c154600 0x7f580c1f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.201+0000 7f5811d20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c0c6f50 0x7f580c1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.201+0000 7f5812521640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c154600 0x7f580c1f0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48120/0 (socket says 172.21.15.133:48120) 2024-02-21T23:26:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.201+0000 7f5812521640 1 -- 172.21.15.133:0/2595644125 learned_addr learned my addr 172.21.15.133:0/2595644125 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:05.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.204+0000 7f581151f640 1 -- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c0c6f50 msgr2=0x7f580c1553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.204+0000 7f581151f640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c0c6f50 0x7f580c1553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.204+0000 7f581151f640 1 -- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c154600 msgr2=0x7f580c1f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.204+0000 7f581151f640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c154600 0x7f580c1f0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.204+0000 7f581151f640 1 -- 172.21.15.133:0/2595644125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c157210 con 0x7f580c0cab80 2024-02-21T23:26:05.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.206+0000 7f581151f640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0cab80 0x7f580c158ee0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f580c0d00a0 tx=0x7f58080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.210+0000 7f58027fc640 1 -- 172.21.15.133:0/2595644125 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808032630 con 0x7f580c0cab80 2024-02-21T23:26:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.210+0000 7f58027fc640 1 -- 172.21.15.133:0/2595644125 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5808032810 con 0x7f580c0cab80 2024-02-21T23:26:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.210+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c1558f0 con 0x7f580c0cab80 2024-02-21T23:26:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.210+0000 7f58027fc640 1 -- 172.21.15.133:0/2595644125 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808032c00 con 0x7f580c0cab80 2024-02-21T23:26:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.210+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c156a80 con 0x7f580c0cab80 2024-02-21T23:26:05.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.211+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580c06a130 con 0x7f580c0cab80 2024-02-21T23:26:05.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.455+0000 7f58027fc640 1 -- 172.21.15.133:0/2595644125 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f580804c060 con 0x7f580c0cab80 2024-02-21T23:26:05.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.455+0000 7f58027fc640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f57f006b520 0x7f57f006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.455+0000 7f5811d20640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f57f006b520 0x7f57f006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.456+0000 7f5811d20640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f57f006b520 0x7f57f006d9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009bd0 tx=0x7f57fc009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.467+0000 7f58027fc640 1 -- 172.21.15.133:0/2595644125 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f580802f030 con 0x7f580c0cab80 2024-02-21T23:26:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.467+0000 7f58027fc640 1 -- 172.21.15.133:0/2595644125 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f5808040070 con 0x7f580c0cab80 2024-02-21T23:26:05.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.644+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f580c0cbd80 con 0x7f580c0cab80 2024-02-21T23:26:05.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.649+0000 7f58027fc640 1 -- 172.21.15.133:0/2595644125 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f580807a1a0 con 0x7f580c0cab80 2024-02-21T23:26:05.635 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f57f006b520 msgr2=0x7f57f006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f57f006b520 0x7f57f006d9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009bd0 tx=0x7f57fc009240 comp rx=0 tx=0).stop 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0cab80 msgr2=0x7f580c158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0cab80 0x7f580c158ee0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f580c0d00a0 tx=0x7f58080046c0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 shutdown_connections 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f57f006b520 0x7f57f006d9e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c154600 0x7f580c1f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c0cab80 0x7f580c158ee0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 --2- 172.21.15.133:0/2595644125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f580c0c6f50 0x7f580c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 >> 172.21.15.133:0/2595644125 conn(0x7f580c062000 msgr2=0x7f580c0ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 shutdown_connections 2024-02-21T23:26:05.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.651+0000 7f5813f84640 1 -- 172.21.15.133:0/2595644125 wait complete. 2024-02-21T23:26:05.648 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-21T23:26:05.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.767+0000 7f8f0898c640 1 Processor -- start 2024-02-21T23:26:05.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.767+0000 7f8f0898c640 1 -- start start 2024-02-21T23:26:05.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.768+0000 7f8f0898c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8f04151000 0x7f8f040701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.768+0000 7f8f0898c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8f04070950 0x7f8f040c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.768+0000 7f8f0898c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f040c7fa0 con 0x7f8f040c4700 2024-02-21T23:26:05.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.768+0000 7f8f0898c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f040c7d70 con 0x7f8f04151000 2024-02-21T23:26:05.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.768+0000 7f8f0898c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f040c6080 con 0x7f8f04070950 2024-02-21T23:26:05.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.768+0000 7f8f02d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8f04070950 0x7f8f040c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.768+0000 7f8f0259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8f04151000 0x7f8f040701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.768+0000 7f8f02d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8f04070950 0x7f8f040c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48130/0 (socket says 172.21.15.133:48130) 2024-02-21T23:26:05.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.768+0000 7f8f02d9d640 1 -- 172.21.15.133:0/3431354595 learned_addr learned my addr 172.21.15.133:0/3431354595 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:05.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.771+0000 7f8f02d9d640 1 -- 172.21.15.133:0/3431354595 >> v2:172.21.15.133:3300/0 conn(0x7f8f04151000 msgr2=0x7f8f040701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:05.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.771+0000 7f8f02d9d640 1 --2- 172.21.15.133:0/3431354595 >> v2:172.21.15.133:3300/0 conn(0x7f8f04151000 0x7f8f040701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.771+0000 7f8f02d9d640 1 -- 172.21.15.133:0/3431354595 >> v1:172.21.15.133:6789/0 conn(0x7f8f040c4700 legacy=0x7f8f040c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.771+0000 7f8f02d9d640 1 -- 172.21.15.133:0/3431354595 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f040c6c20 con 0x7f8f04070950 2024-02-21T23:26:05.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.773+0000 7f8f02d9d640 1 --2- 172.21.15.133:0/3431354595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f04070950 0x7f8f040c7150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f040ca2f0 tx=0x7f8efc02e180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c1c6bdfab6b2d3d server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.776+0000 7f8f01d9b640 1 -- 172.21.15.133:0/3431354595 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00b0d0 con 0x7f8f04070950 2024-02-21T23:26:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.776+0000 7f8f01d9b640 1 -- 172.21.15.133:0/3431354595 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00b270 con 0x7f8f04070950 2024-02-21T23:26:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.776+0000 7f8f01d9b640 1 -- 172.21.15.133:0/3431354595 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc009600 con 0x7f8f04070950 2024-02-21T23:26:05.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.777+0000 7f8f0898c640 1 -- 172.21.15.133:0/3431354595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f04070950 msgr2=0x7f8f040c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.777+0000 7f8f0898c640 1 --2- 172.21.15.133:0/3431354595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f04070950 0x7f8f040c7150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f040ca2f0 tx=0x7f8efc02e180 comp rx=0 tx=0).stop 2024-02-21T23:26:05.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.777+0000 7f8f0898c640 1 -- 172.21.15.133:0/3431354595 shutdown_connections 2024-02-21T23:26:05.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.777+0000 7f8f0898c640 1 --2- 172.21.15.133:0/3431354595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f04070950 0x7f8f040c7150 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.777+0000 7f8f0898c640 1 --2- 172.21.15.133:0/3431354595 >> v2:172.21.15.133:3300/0 conn(0x7f8f04151000 0x7f8f040701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.777+0000 7f8f0898c640 1 -- 172.21.15.133:0/3431354595 >> 172.21.15.133:0/3431354595 conn(0x7f8f0406bfd0 msgr2=0x7f8f0406c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:05.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.777+0000 7f8f0898c640 1 -- 172.21.15.133:0/3431354595 shutdown_connections 2024-02-21T23:26:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.777+0000 7f8f0898c640 1 -- 172.21.15.133:0/3431354595 wait complete. 2024-02-21T23:26:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.778+0000 7f8f0898c640 1 Processor -- start 2024-02-21T23:26:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.778+0000 7f8f0898c640 1 -- start start 2024-02-21T23:26:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.779+0000 7f8f0898c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f04070950 0x7f8f041504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.779+0000 7f8f0898c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f04151000 0x7f8f040cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.779+0000 7f8f0898c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0414fd20 0x7f8f0415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.779+0000 7f8f0898c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040c8820 con 0x7f8f04070950 2024-02-21T23:26:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.779+0000 7f8f0898c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040c8540 con 0x7f8f0414fd20 2024-02-21T23:26:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.779+0000 7f8f0898c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040706e0 con 0x7f8f04151000 2024-02-21T23:26:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.780+0000 7f8f02d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0414fd20 0x7f8f0415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.780+0000 7f8f0259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f04070950 0x7f8f041504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.780+0000 7f8efbfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f04151000 0x7f8f040cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.780+0000 7f8f02d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0414fd20 0x7f8f0415a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48134/0 (socket says 172.21.15.133:48134) 2024-02-21T23:26:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.780+0000 7f8f02d9d640 1 -- 172.21.15.133:0/2262517446 learned_addr learned my addr 172.21.15.133:0/2262517446 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:05.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.783+0000 7f8f02d9d640 1 -- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f04151000 msgr2=0x7f8f040cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.783+0000 7f8f02d9d640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f04151000 0x7f8f040cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.783+0000 7f8f02d9d640 1 -- 172.21.15.133:0/2262517446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f04070950 msgr2=0x7f8f041504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:05.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.783+0000 7f8f02d9d640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f04070950 0x7f8f041504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:05.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.783+0000 7f8f02d9d640 1 -- 172.21.15.133:0/2262517446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0415ad30 con 0x7f8f0414fd20 2024-02-21T23:26:05.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.783+0000 7f8f0259c640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f04070950 0x7f8f041504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:26:05.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.785+0000 7f8f02d9d640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0414fd20 0x7f8f0415a330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00d240 tx=0x7f8efc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.789+0000 7f8efaffd640 1 -- 172.21.15.133:0/2262517446 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc045020 con 0x7f8f0414fd20 2024-02-21T23:26:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.789+0000 7f8efaffd640 1 -- 172.21.15.133:0/2262517446 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00b420 con 0x7f8f0414fd20 2024-02-21T23:26:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.789+0000 7f8efaffd640 1 -- 172.21.15.133:0/2262517446 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0408a0 con 0x7f8f0414fd20 2024-02-21T23:26:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.789+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f04158180 con 0x7f8f0414fd20 2024-02-21T23:26:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.789+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0415afd0 con 0x7f8f0414fd20 2024-02-21T23:26:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:05.790+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0406a130 con 0x7f8f0414fd20 2024-02-21T23:26:06.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.042+0000 7f8efaffd640 1 -- 172.21.15.133:0/2262517446 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8efc048020 con 0x7f8f0414fd20 2024-02-21T23:26:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.043+0000 7f8efaffd640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ee406b550 0x7f8ee406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.043+0000 7f8f0259c640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ee406b550 0x7f8ee406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.043+0000 7f8f0259c640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ee406b550 0x7f8ee406da10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e90 tx=0x7f8ef4009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:06.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.045+0000 7f8efaffd640 1 -- 172.21.15.133:0/2262517446 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f8efc07b740 con 0x7f8f0414fd20 2024-02-21T23:26:06.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.058+0000 7f8efaffd640 1 -- 172.21.15.133:0/2262517446 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f8efc04f030 con 0x7f8f0414fd20 2024-02-21T23:26:06.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.231+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8f04068060 con 0x7f8f0414fd20 2024-02-21T23:26:06.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.236+0000 7f8efaffd640 1 -- 172.21.15.133:0/2262517446 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8efc07d6a0 con 0x7f8f0414fd20 2024-02-21T23:26:06.223 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:26:06.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ee406b550 msgr2=0x7f8ee406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ee406b550 0x7f8ee406da10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e90 tx=0x7f8ef4009190 comp rx=0 tx=0).stop 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0414fd20 msgr2=0x7f8f0415a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0414fd20 0x7f8f0415a330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00d240 tx=0x7f8efc004a40 comp rx=0 tx=0).stop 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 shutdown_connections 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8ee406b550 0x7f8ee406da10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0414fd20 0x7f8f0415a330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f04151000 0x7f8f040cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 --2- 172.21.15.133:0/2262517446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f04070950 0x7f8f041504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 >> 172.21.15.133:0/2262517446 conn(0x7f8f0406bfd0 msgr2=0x7f8f0406c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 shutdown_connections 2024-02-21T23:26:06.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.238+0000 7f8f0898c640 1 -- 172.21.15.133:0/2262517446 wait complete. 2024-02-21T23:26:06.237 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:26:06.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.349+0000 7f3fa5d8c640 1 Processor -- start 2024-02-21T23:26:06.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.349+0000 7f3fa5d8c640 1 -- start start 2024-02-21T23:26:06.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.350+0000 7f3fa5d8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9830 0x7f3fa0150940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.350+0000 7f3fa5d8c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa0073210 con 0x7f3fa00c9450 2024-02-21T23:26:06.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.350+0000 7f3fa5d8c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa015a280 con 0x7f3fa00c7790 2024-02-21T23:26:06.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.350+0000 7f3fa5d8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa015a560 con 0x7f3fa00c9830 2024-02-21T23:26:06.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.351+0000 7f3f9effd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f3fa00c7790 0x7f3fa015d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:41488/0 (socket says 172.21.15.133:41488) 2024-02-21T23:26:06.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.351+0000 7f3f9effd640 1 -- 172.21.15.133:0/3843051874 learned_addr learned my addr 172.21.15.133:0/3843051874 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:26:06.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.351+0000 7f3f9ffff640 1 --2- 172.21.15.133:0/3843051874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9830 0x7f3fa0150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.353+0000 7f3f9e7fc640 1 -- 172.21.15.133:0/3843051874 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667657546 0 0) 0x7f3fa0073210 con 0x7f3fa00c9450 2024-02-21T23:26:06.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.353+0000 7f3f9e7fc640 1 -- 172.21.15.133:0/3843051874 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f88003520 con 0x7f3fa00c9450 2024-02-21T23:26:06.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.353+0000 7f3f9e7fc640 1 -- 172.21.15.133:0/3843051874 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699794620 0 0) 0x7f3fa015a280 con 0x7f3fa00c7790 2024-02-21T23:26:06.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.353+0000 7f3f9e7fc640 1 -- 172.21.15.133:0/3843051874 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa0073210 con 0x7f3fa00c7790 2024-02-21T23:26:06.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.354+0000 7f3f9ffff640 1 -- 172.21.15.133:0/3843051874 >> v1:172.21.15.142:6789/0 conn(0x7f3fa00c7790 legacy=0x7f3fa015d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.354+0000 7f3f9ffff640 1 -- 172.21.15.133:0/3843051874 >> v1:172.21.15.133:6789/0 conn(0x7f3fa00c9450 legacy=0x7f3fa0072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.354+0000 7f3f9ffff640 1 -- 172.21.15.133:0/3843051874 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa015a840 con 0x7f3fa00c9830 2024-02-21T23:26:06.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.356+0000 7f3f9ffff640 1 --2- 172.21.15.133:0/3843051874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9830 0x7f3fa0150940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009dc0 tx=0x7f3f9802bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce5bf2bab409c44 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:06.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.362+0000 7f3f9e7fc640 1 -- 172.21.15.133:0/3843051874 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98039070 con 0x7f3fa00c9830 2024-02-21T23:26:06.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.362+0000 7f3f9e7fc640 1 -- 172.21.15.133:0/3843051874 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9802cd50 con 0x7f3fa00c9830 2024-02-21T23:26:06.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.362+0000 7f3f9e7fc640 1 -- 172.21.15.133:0/3843051874 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98034660 con 0x7f3fa00c9830 2024-02-21T23:26:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.363+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/3843051874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9830 msgr2=0x7f3fa0150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.363+0000 7f3fa5d8c640 1 --2- 172.21.15.133:0/3843051874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9830 0x7f3fa0150940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009dc0 tx=0x7f3f9802bfa0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.363+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/3843051874 shutdown_connections 2024-02-21T23:26:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.363+0000 7f3fa5d8c640 1 --2- 172.21.15.133:0/3843051874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9830 0x7f3fa0150940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.363+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/3843051874 >> 172.21.15.133:0/3843051874 conn(0x7f3fa006bfd0 msgr2=0x7f3fa006c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.363+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/3843051874 shutdown_connections 2024-02-21T23:26:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.363+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/3843051874 wait complete. 2024-02-21T23:26:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.365+0000 7f3fa5d8c640 1 Processor -- start 2024-02-21T23:26:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.365+0000 7f3fa5d8c640 1 -- start start 2024-02-21T23:26:06.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.366+0000 7f3fa5d8c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa00c7790 0x7f3fa00c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.366+0000 7f3fa5d8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9450 0x7f3fa015c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.366+0000 7f3fa5d8c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa00c9830 0x7f3fa01456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.366+0000 7f3fa5d8c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa015dff0 con 0x7f3fa00c9830 2024-02-21T23:26:06.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.366+0000 7f3fa5d8c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa015de70 con 0x7f3fa00c7790 2024-02-21T23:26:06.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.366+0000 7f3fa5d8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00cbfb0 con 0x7f3fa00c9450 2024-02-21T23:26:06.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.366+0000 7f3f9f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa00c7790 0x7f3fa00c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.367+0000 7f3f9effd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9450 0x7f3fa015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.367+0000 7f3f9f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa00c7790 0x7f3fa00c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48136/0 (socket says 172.21.15.133:48136) 2024-02-21T23:26:06.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.367+0000 7f3f9f7fe640 1 -- 172.21.15.133:0/427736384 learned_addr learned my addr 172.21.15.133:0/427736384 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:06.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.368+0000 7f3f9ffff640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa00c9830 0x7f3fa01456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.370+0000 7f3f9f7fe640 1 -- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9450 msgr2=0x7f3fa015c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.370+0000 7f3f9f7fe640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9450 0x7f3fa015c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.370+0000 7f3f9f7fe640 1 -- 172.21.15.133:0/427736384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa00c9830 msgr2=0x7f3fa01456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.370+0000 7f3f9f7fe640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa00c9830 0x7f3fa01456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.370+0000 7f3f9f7fe640 1 -- 172.21.15.133:0/427736384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00cc290 con 0x7f3fa00c7790 2024-02-21T23:26:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.372+0000 7f3f9f7fe640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa00c7790 0x7f3fa00c6da0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000c460 tx=0x7f3f9000c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:06.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.376+0000 7f3f87fff640 1 -- 172.21.15.133:0/427736384 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f900052d0 con 0x7f3fa00c7790 2024-02-21T23:26:06.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.376+0000 7f3f87fff640 1 -- 172.21.15.133:0/427736384 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f90005470 con 0x7f3fa00c7790 2024-02-21T23:26:06.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.376+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa00c5d20 con 0x7f3fa00c7790 2024-02-21T23:26:06.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.376+0000 7f3f87fff640 1 -- 172.21.15.133:0/427736384 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90012850 con 0x7f3fa00c7790 2024-02-21T23:26:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.377+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa0159f70 con 0x7f3fa00c7790 2024-02-21T23:26:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.377+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa006a130 con 0x7f3fa00c7790 2024-02-21T23:26:06.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.635+0000 7f3f87fff640 1 -- 172.21.15.133:0/427736384 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f900129f0 con 0x7f3fa00c7790 2024-02-21T23:26:06.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.635+0000 7f3f87fff640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3f8c06b520 0x7f3f8c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.636+0000 7f3f9effd640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3f8c06b520 0x7f3f8c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.636+0000 7f3f9effd640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3f8c06b520 0x7f3f8c06d9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3fa014d110 tx=0x7f3f94009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:06.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.638+0000 7f3f87fff640 1 -- 172.21.15.133:0/427736384 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f3f900570b0 con 0x7f3fa00c7790 2024-02-21T23:26:06.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.650+0000 7f3f87fff640 1 -- 172.21.15.133:0/427736384 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f3f90059010 con 0x7f3fa00c7790 2024-02-21T23:26:06.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.866+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fa00c7ce0 con 0x7f3fa00c7790 2024-02-21T23:26:06.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.881+0000 7f3f87fff640 1 -- 172.21.15.133:0/427736384 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6535 (secure 0 0 0) 0x7f3f9005d0b0 con 0x7f3fa00c7790 2024-02-21T23:26:06.867 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:26:06.868 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1319340142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1319340142},{"type":"v1","addr":"172.21.15.133:6837","nonce":1319340142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1740931674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1740931674},{"type":"v1","addr":"172.21.15.142:6837","nonce":1740931674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3712864993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3712864993},{"type":"v1","addr":"172.21.15.142:6839","nonce":3712864993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2928905924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2928905924},{"type":"v1","addr":"172.21.15.133:6833","nonce":2928905924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3461010406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3461010406},{"type":"v1","addr":"172.21.15.133:6835","nonce":3461010406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-21T23:25:16.775942+0000","modified":"2024-02-21T23:25:49.141553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6835/1527124199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1527124199},{"type":"v1","addr":"172.21.15.142:6835","nonce":1527124199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-21T23:26:06.868 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2024-02-21T23:26:06.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3f8c06b520 msgr2=0x7f3f8c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3f8c06b520 0x7f3f8c06d9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3fa014d110 tx=0x7f3f94009040 comp rx=0 tx=0).stop 2024-02-21T23:26:06.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa00c7790 msgr2=0x7f3fa00c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa00c7790 0x7f3fa00c6da0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000c460 tx=0x7f3f9000c920 comp rx=0 tx=0).stop 2024-02-21T23:26:06.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 shutdown_connections 2024-02-21T23:26:06.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f3f8c06b520 0x7f3f8c06d9e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa00c9830 0x7f3fa01456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa00c9450 0x7f3fa015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 --2- 172.21.15.133:0/427736384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa00c7790 0x7f3fa00c6da0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 >> 172.21.15.133:0/427736384 conn(0x7f3fa006bfd0 msgr2=0x7f3fa006c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:06.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 shutdown_connections 2024-02-21T23:26:06.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.883+0000 7f3fa5d8c640 1 -- 172.21.15.133:0/427736384 wait complete. 2024-02-21T23:26:06.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:26:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.995+0000 7f976e7aa640 1 Processor -- start 2024-02-21T23:26:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.995+0000 7f976e7aa640 1 -- start start 2024-02-21T23:26:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.996+0000 7f976e7aa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f97680ce3c0 0x7f97680c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.996+0000 7f976e7aa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 0x7f976814e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.996+0000 7f976e7aa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976814eea0 con 0x7f97680cdfe0 2024-02-21T23:26:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.996+0000 7f976e7aa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97680c84b0 con 0x7f97680ce3c0 2024-02-21T23:26:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.996+0000 7f976e7aa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9768158ca0 con 0x7f976814e510 2024-02-21T23:26:06.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.997+0000 7f976cd47640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 0x7f976814e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.997+0000 7f97677fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f97680ce3c0 0x7f97680c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.997+0000 7f9767fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f97680cdfe0 0x7f9768070d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45190/0 (socket says 172.21.15.133:45190) 2024-02-21T23:26:06.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.997+0000 7f9767fff640 1 -- 172.21.15.133:0/593889403 learned_addr learned my addr 172.21.15.133:0/593889403 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:26:06.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.999+0000 7f9766ffd640 1 -- 172.21.15.133:0/593889403 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165315598 0 0) 0x7f976814eea0 con 0x7f97680cdfe0 2024-02-21T23:26:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:06.999+0000 7f9766ffd640 1 -- 172.21.15.133:0/593889403 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9750003520 con 0x7f97680cdfe0 2024-02-21T23:26:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.000+0000 7f976cd47640 1 -- 172.21.15.133:0/593889403 >> v2:172.21.15.142:3300/0 conn(0x7f97680ce3c0 msgr2=0x7f97680c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.000+0000 7f976cd47640 1 --2- 172.21.15.133:0/593889403 >> v2:172.21.15.142:3300/0 conn(0x7f97680ce3c0 0x7f97680c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.000+0000 7f976cd47640 1 -- 172.21.15.133:0/593889403 >> v1:172.21.15.133:6789/0 conn(0x7f97680cdfe0 legacy=0x7f9768070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.000+0000 7f976cd47640 1 -- 172.21.15.133:0/593889403 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97680717e0 con 0x7f976814e510 2024-02-21T23:26:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.002+0000 7f976cd47640 1 --2- 172.21.15.133:0/593889403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 0x7f976814e960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9768158e20 tx=0x7f976002be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8529e38eee4596 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:06.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.005+0000 7f9766ffd640 1 -- 172.21.15.133:0/593889403 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9760039070 con 0x7f976814e510 2024-02-21T23:26:06.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.007+0000 7f9766ffd640 1 -- 172.21.15.133:0/593889403 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976002cc40 con 0x7f976814e510 2024-02-21T23:26:06.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.007+0000 7f9766ffd640 1 -- 172.21.15.133:0/593889403 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9760034440 con 0x7f976814e510 2024-02-21T23:26:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.008+0000 7f976e7aa640 1 -- 172.21.15.133:0/593889403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 msgr2=0x7f976814e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.008+0000 7f976e7aa640 1 --2- 172.21.15.133:0/593889403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 0x7f976814e960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9768158e20 tx=0x7f976002be90 comp rx=0 tx=0).stop 2024-02-21T23:26:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.008+0000 7f976e7aa640 1 -- 172.21.15.133:0/593889403 shutdown_connections 2024-02-21T23:26:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.008+0000 7f976e7aa640 1 --2- 172.21.15.133:0/593889403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 0x7f976814e960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.008+0000 7f976e7aa640 1 --2- 172.21.15.133:0/593889403 >> v2:172.21.15.142:3300/0 conn(0x7f97680ce3c0 0x7f97680c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.008+0000 7f976e7aa640 1 -- 172.21.15.133:0/593889403 >> 172.21.15.133:0/593889403 conn(0x7f976806aec0 msgr2=0x7f976806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.008+0000 7f976e7aa640 1 -- 172.21.15.133:0/593889403 shutdown_connections 2024-02-21T23:26:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.008+0000 7f976e7aa640 1 -- 172.21.15.133:0/593889403 wait complete. 2024-02-21T23:26:06.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.009+0000 7f976e7aa640 1 Processor -- start 2024-02-21T23:26:06.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.009+0000 7f976e7aa640 1 -- start start 2024-02-21T23:26:06.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.010+0000 7f976e7aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97680cdfe0 0x7f976814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.010+0000 7f976e7aa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97680ce3c0 0x7f97680adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.010+0000 7f976e7aa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 0x7f97680ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:06.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.010+0000 7f976e7aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768158650 con 0x7f97680cdfe0 2024-02-21T23:26:06.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.010+0000 7f976e7aa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768158370 con 0x7f97680ce3c0 2024-02-21T23:26:06.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.010+0000 7f976e7aa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9768158930 con 0x7f976814e510 2024-02-21T23:26:06.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.010+0000 7f9767fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97680cdfe0 0x7f976814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.010+0000 7f97677fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97680ce3c0 0x7f97680adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.011+0000 7f976cd47640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 0x7f97680ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:06.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.011+0000 7f9767fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97680cdfe0 0x7f976814b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46146/0 (socket says 172.21.15.133:46146) 2024-02-21T23:26:06.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.011+0000 7f9767fff640 1 -- 172.21.15.133:0/4083618483 learned_addr learned my addr 172.21.15.133:0/4083618483 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:07.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.014+0000 7f9767fff640 1 -- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 msgr2=0x7f97680ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:07.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.014+0000 7f9767fff640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 0x7f97680ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.014+0000 7f9767fff640 1 -- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97680ce3c0 msgr2=0x7f97680adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:07.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.014+0000 7f9767fff640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97680ce3c0 0x7f97680adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.014+0000 7f9767fff640 1 -- 172.21.15.133:0/4083618483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97680b0600 con 0x7f97680cdfe0 2024-02-21T23:26:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.016+0000 7f9767fff640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97680cdfe0 0x7f976814b090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f975800c4d0 tx=0x7f975800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:07.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.020+0000 7f9764ff9640 1 -- 172.21.15.133:0/4083618483 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758017070 con 0x7f97680cdfe0 2024-02-21T23:26:07.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.020+0000 7f9764ff9640 1 -- 172.21.15.133:0/4083618483 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9758005d00 con 0x7f97680cdfe0 2024-02-21T23:26:07.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.020+0000 7f9764ff9640 1 -- 172.21.15.133:0/4083618483 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97580072b0 con 0x7f97680cdfe0 2024-02-21T23:26:07.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.020+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97680b0360 con 0x7f97680cdfe0 2024-02-21T23:26:07.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.020+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97680c9b40 con 0x7f97680cdfe0 2024-02-21T23:26:07.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.021+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976806a130 con 0x7f97680cdfe0 2024-02-21T23:26:07.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.038+0000 7f9764ff9640 1 -- 172.21.15.133:0/4083618483 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f975801a070 con 0x7f97680cdfe0 2024-02-21T23:26:07.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.038+0000 7f9764ff9640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f975406b520 0x7f975406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:07.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.038+0000 7f97677fe640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f975406b520 0x7f975406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:07.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.039+0000 7f97677fe640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f975406b520 0x7f975406d9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9768159e50 tx=0x7f975c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.041+0000 7f9764ff9640 1 -- 172.21.15.133:0/4083618483 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f9758056260 con 0x7f97680cdfe0 2024-02-21T23:26:07.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.278+0000 7f9764ff9640 1 -- 172.21.15.133:0/4083618483 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f9758021020 con 0x7f97680cdfe0 2024-02-21T23:26:07.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.451+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f976814cf00 con 0x7f97680cdfe0 2024-02-21T23:26:07.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.497+0000 7f9764ff9640 1 -- 172.21.15.133:0/4083618483 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19091 (secure 0 0 0) 0x7f97580581c0 con 0x7f97680cdfe0 2024-02-21T23:26:07.484 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:26:07.485 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:57.029812+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":0}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:26:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f975406b520 msgr2=0x7f975406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f975406b520 0x7f975406d9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9768159e50 tx=0x7f975c009190 comp rx=0 tx=0).stop 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97680cdfe0 msgr2=0x7f976814b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97680cdfe0 0x7f976814b090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f975800c4d0 tx=0x7f975800c990 comp rx=0 tx=0).stop 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 shutdown_connections 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f975406b520 0x7f975406d9e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f976814e510 0x7f97680ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97680ce3c0 0x7f97680adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 --2- 172.21.15.133:0/4083618483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97680cdfe0 0x7f976814b090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.499+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 >> 172.21.15.133:0/4083618483 conn(0x7f976806aec0 msgr2=0x7f9768156590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.500+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 shutdown_connections 2024-02-21T23:26:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.500+0000 7f976e7aa640 1 -- 172.21.15.133:0/4083618483 wait complete. 2024-02-21T23:26:07.498 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:26:07.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.612+0000 7fe2d851e640 1 Processor -- start 2024-02-21T23:26:07.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.612+0000 7fe2d851e640 1 -- start start 2024-02-21T23:26:07.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.612+0000 7fe2d851e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe2d0150f80 0x7fe2d015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:07.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.613+0000 7fe2d851e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe2d015b020 0x7fe2d00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:07.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.613+0000 7fe2d851e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d00c2d70 con 0x7fe2d00cbb30 2024-02-21T23:26:07.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.613+0000 7fe2d851e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d00c2b40 con 0x7fe2d0150f80 2024-02-21T23:26:07.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.613+0000 7fe2d851e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d00c3050 con 0x7fe2d015b020 2024-02-21T23:26:07.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.613+0000 7fe2d62ba640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe2d0150f80 0x7fe2d015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:07.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.613+0000 7fe2d5ab9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe2d015b020 0x7fe2d00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:07.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.613+0000 7fe2d5ab9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe2d015b020 0x7fe2d00cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48162/0 (socket says 172.21.15.133:48162) 2024-02-21T23:26:07.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.614+0000 7fe2d5ab9640 1 -- 172.21.15.133:0/839620338 learned_addr learned my addr 172.21.15.133:0/839620338 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:07.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.616+0000 7fe2d5ab9640 1 -- 172.21.15.133:0/839620338 >> v2:172.21.15.133:3300/0 conn(0x7fe2d0150f80 msgr2=0x7fe2d015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:07.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.616+0000 7fe2d5ab9640 1 --2- 172.21.15.133:0/839620338 >> v2:172.21.15.133:3300/0 conn(0x7fe2d0150f80 0x7fe2d015aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.616+0000 7fe2d5ab9640 1 -- 172.21.15.133:0/839620338 >> v1:172.21.15.133:6789/0 conn(0x7fe2d00cbb30 legacy=0x7fe2d00c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:07.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.616+0000 7fe2d5ab9640 1 -- 172.21.15.133:0/839620338 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d00c3330 con 0x7fe2d015b020 2024-02-21T23:26:07.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.617+0000 7fe2d5ab9640 1 --2- 172.21.15.133:0/839620338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d015b020 0x7fe2d00cb5f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0071680 tx=0x7fe2c002ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f983fdcf2b0c5e8f server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:07.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.621+0000 7fe2d52b8640 1 -- 172.21.15.133:0/839620338 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c002f850 con 0x7fe2d015b020 2024-02-21T23:26:07.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.621+0000 7fe2d52b8640 1 -- 172.21.15.133:0/839620338 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c002f9f0 con 0x7fe2d015b020 2024-02-21T23:26:07.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.621+0000 7fe2d52b8640 1 -- 172.21.15.133:0/839620338 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c002fdc0 con 0x7fe2d015b020 2024-02-21T23:26:07.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.622+0000 7fe2d851e640 1 -- 172.21.15.133:0/839620338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d015b020 msgr2=0x7fe2d00cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:07.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.622+0000 7fe2d851e640 1 --2- 172.21.15.133:0/839620338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d015b020 0x7fe2d00cb5f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0071680 tx=0x7fe2c002ba30 comp rx=0 tx=0).stop 2024-02-21T23:26:07.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.622+0000 7fe2d851e640 1 -- 172.21.15.133:0/839620338 shutdown_connections 2024-02-21T23:26:07.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.622+0000 7fe2d851e640 1 --2- 172.21.15.133:0/839620338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d015b020 0x7fe2d00cb5f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.622+0000 7fe2d851e640 1 --2- 172.21.15.133:0/839620338 >> v2:172.21.15.133:3300/0 conn(0x7fe2d0150f80 0x7fe2d015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.622+0000 7fe2d851e640 1 -- 172.21.15.133:0/839620338 >> 172.21.15.133:0/839620338 conn(0x7fe2d006aec0 msgr2=0x7fe2d006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:07.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.622+0000 7fe2d851e640 1 -- 172.21.15.133:0/839620338 shutdown_connections 2024-02-21T23:26:07.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.622+0000 7fe2d851e640 1 -- 172.21.15.133:0/839620338 wait complete. 2024-02-21T23:26:07.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.623+0000 7fe2d851e640 1 Processor -- start 2024-02-21T23:26:07.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.623+0000 7fe2d851e640 1 -- start start 2024-02-21T23:26:07.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.624+0000 7fe2d851e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2d00cbb30 0x7fe2d014c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:07.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.624+0000 7fe2d851e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d0150f80 0x7fe2d01494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:07.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.624+0000 7fe2d851e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d015b020 0x7fe2d0141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:07.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.624+0000 7fe2d851e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00c5990 con 0x7fe2d015b020 2024-02-21T23:26:07.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.624+0000 7fe2d851e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d015a5c0 con 0x7fe2d0150f80 2024-02-21T23:26:07.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.624+0000 7fe2d851e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00c5b10 con 0x7fe2d00cbb30 2024-02-21T23:26:07.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.624+0000 7fe2d5ab9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d0150f80 0x7fe2d01494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:07.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.624+0000 7fe2d6abb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d015b020 0x7fe2d0141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:07.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.625+0000 7fe2d5ab9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d0150f80 0x7fe2d01494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48168/0 (socket says 172.21.15.133:48168) 2024-02-21T23:26:07.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.625+0000 7fe2d5ab9640 1 -- 172.21.15.133:0/3135196169 learned_addr learned my addr 172.21.15.133:0/3135196169 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:07.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.625+0000 7fe2d62ba640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2d00cbb30 0x7fe2d014c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.628+0000 7fe2d6abb640 1 -- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2d00cbb30 msgr2=0x7fe2d014c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.628+0000 7fe2d6abb640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2d00cbb30 0x7fe2d014c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.628+0000 7fe2d6abb640 1 -- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d0150f80 msgr2=0x7fe2d01494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.628+0000 7fe2d6abb640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d0150f80 0x7fe2d01494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.628+0000 7fe2d6abb640 1 -- 172.21.15.133:0/3135196169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d014cde0 con 0x7fe2d015b020 2024-02-21T23:26:07.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.630+0000 7fe2d6abb640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d015b020 0x7fe2d0141ed0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c6c0 tx=0x7fe2c800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:07.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.634+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3135196169 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8003b70 con 0x7fe2d015b020 2024-02-21T23:26:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.634+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3135196169 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c8003d10 con 0x7fe2d015b020 2024-02-21T23:26:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.634+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3135196169 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80052f0 con 0x7fe2d015b020 2024-02-21T23:26:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.634+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d01f3e80 con 0x7fe2d015b020 2024-02-21T23:26:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.634+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d01f4140 con 0x7fe2d015b020 2024-02-21T23:26:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.635+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d006a130 con 0x7fe2d015b020 2024-02-21T23:26:07.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.652+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3135196169 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2c8005490 con 0x7fe2d015b020 2024-02-21T23:26:07.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.653+0000 7fe2c6ffd640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2b406b4d0 0x7fe2b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:07.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.653+0000 7fe2d62ba640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2b406b4d0 0x7fe2b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:07.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.653+0000 7fe2d62ba640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2b406b4d0 0x7fe2b406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009b40 tx=0x7fe2cc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:07.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.891+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3135196169 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fe2c8057290 con 0x7fe2d015b020 2024-02-21T23:26:07.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:07.892+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3135196169 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fe2c80591f0 con 0x7fe2d015b020 2024-02-21T23:26:08.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.119+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2d00c43a0 con 0x7fe2d015b020 2024-02-21T23:26:08.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.132+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3135196169 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6535 (secure 0 0 0) 0x7fe2c805f070 con 0x7fe2d015b020 2024-02-21T23:26:08.119 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:26:08.119 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1319340142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1319340142},{"type":"v1","addr":"172.21.15.133:6837","nonce":1319340142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1740931674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1740931674},{"type":"v1","addr":"172.21.15.142:6837","nonce":1740931674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3712864993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3712864993},{"type":"v1","addr":"172.21.15.142:6839","nonce":3712864993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2928905924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2928905924},{"type":"v1","addr":"172.21.15.133:6833","nonce":2928905924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3461010406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3461010406},{"type":"v1","addr":"172.21.15.133:6835","nonce":3461010406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-21T23:25:16.775942+0000","modified":"2024-02-21T23:25:49.141553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6835/1527124199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1527124199},{"type":"v1","addr":"172.21.15.142:6835","nonce":1527124199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-21T23:26:08.119 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2024-02-21T23:26:08.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2b406b4d0 msgr2=0x7fe2b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:08.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2b406b4d0 0x7fe2b406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009b40 tx=0x7fe2cc009190 comp rx=0 tx=0).stop 2024-02-21T23:26:08.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d015b020 msgr2=0x7fe2d0141ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:08.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d015b020 0x7fe2d0141ed0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c6c0 tx=0x7fe2c800cb80 comp rx=0 tx=0).stop 2024-02-21T23:26:08.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 shutdown_connections 2024-02-21T23:26:08.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2b406b4d0 0x7fe2b406d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d015b020 0x7fe2d0141ed0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d0150f80 0x7fe2d01494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 --2- 172.21.15.133:0/3135196169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2d00cbb30 0x7fe2d014c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 >> 172.21.15.133:0/3135196169 conn(0x7fe2d006aec0 msgr2=0x7fe2d00c4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:08.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 shutdown_connections 2024-02-21T23:26:08.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.134+0000 7fe2d851e640 1 -- 172.21.15.133:0/3135196169 wait complete. 2024-02-21T23:26:08.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-21T23:25:16.775942+0000', 'modified': '2024-02-21T23:25:49.141553+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4269}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4269': {'gid': 4269, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.142:6835/1527124199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 1527124199}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 1527124199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-21T23:26:08.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-21T23:26:08.737 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-02-21T23:26:08.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.852+0000 7f6b0340f640 1 Processor -- start 2024-02-21T23:26:08.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.853+0000 7f6b0340f640 1 -- start start 2024-02-21T23:26:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.853+0000 7f6b0340f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc154600 0x7f6afc0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.853+0000 7f6b0340f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6afc0c6f50 0x7f6afc0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.853+0000 7f6b0340f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6afc0cab80 0x7f6afc0c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.853+0000 7f6b0340f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6afc0c74a0 con 0x7f6afc0cab80 2024-02-21T23:26:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.853+0000 7f6b0340f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6afc0c7780 con 0x7f6afc0c6f50 2024-02-21T23:26:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.853+0000 7f6b0340f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc15ab40 con 0x7f6afc154600 2024-02-21T23:26:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.854+0000 7f6b019ac640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6afc0cab80 0x7f6afc0c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.854+0000 7f6b011ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc154600 0x7f6afc0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.854+0000 7f6b009aa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6afc0c6f50 0x7f6afc0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.854+0000 7f6b011ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc154600 0x7f6afc0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:52810/0 (socket says 172.21.15.133:52810) 2024-02-21T23:26:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.854+0000 7f6b019ac640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6afc0cab80 0x7f6afc0c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46182/0 (socket says 172.21.15.133:46182) 2024-02-21T23:26:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.854+0000 7f6b011ab640 1 -- 172.21.15.133:0/322319487 learned_addr learned my addr 172.21.15.133:0/322319487 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:08.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.857+0000 7f6b011ab640 1 -- 172.21.15.133:0/322319487 >> v2:172.21.15.142:3300/0 conn(0x7f6afc0c6f50 msgr2=0x7f6afc0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.857+0000 7f6b011ab640 1 --2- 172.21.15.133:0/322319487 >> v2:172.21.15.142:3300/0 conn(0x7f6afc0c6f50 0x7f6afc0ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.857+0000 7f6b011ab640 1 -- 172.21.15.133:0/322319487 >> v2:172.21.15.133:3300/0 conn(0x7f6afc0cab80 msgr2=0x7f6afc0c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.857+0000 7f6b011ab640 1 --2- 172.21.15.133:0/322319487 >> v2:172.21.15.133:3300/0 conn(0x7f6afc0cab80 0x7f6afc0c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.857+0000 7f6b011ab640 1 -- 172.21.15.133:0/322319487 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc15ae00 con 0x7f6afc154600 2024-02-21T23:26:08.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.857+0000 7f6b019ac640 1 --2- 172.21.15.133:0/322319487 >> v2:172.21.15.133:3300/0 conn(0x7f6afc0cab80 0x7f6afc0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:26:08.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.859+0000 7f6b011ab640 1 --2- 172.21.15.133:0/322319487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc154600 0x7f6afc0cbc00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6afc06d2a0 tx=0x7f6af402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ad278949493392 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:08.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6af3fff640 1 -- 172.21.15.133:0/322319487 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af402ca70 con 0x7f6afc154600 2024-02-21T23:26:08.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6af3fff640 1 -- 172.21.15.133:0/322319487 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af402cc10 con 0x7f6afc154600 2024-02-21T23:26:08.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6af3fff640 1 -- 172.21.15.133:0/322319487 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af40343f0 con 0x7f6afc154600 2024-02-21T23:26:08.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6b0340f640 1 -- 172.21.15.133:0/322319487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc154600 msgr2=0x7f6afc0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:08.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6b0340f640 1 --2- 172.21.15.133:0/322319487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc154600 0x7f6afc0cbc00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6afc06d2a0 tx=0x7f6af402bed0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6b0340f640 1 -- 172.21.15.133:0/322319487 shutdown_connections 2024-02-21T23:26:08.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6b0340f640 1 --2- 172.21.15.133:0/322319487 >> v2:172.21.15.133:3300/0 conn(0x7f6afc0cab80 0x7f6afc0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6b0340f640 1 --2- 172.21.15.133:0/322319487 >> v2:172.21.15.142:3300/0 conn(0x7f6afc0c6f50 0x7f6afc0ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6b0340f640 1 --2- 172.21.15.133:0/322319487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc154600 0x7f6afc0cbc00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6b0340f640 1 -- 172.21.15.133:0/322319487 >> 172.21.15.133:0/322319487 conn(0x7f6afc062000 msgr2=0x7f6afc0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:08.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6b0340f640 1 -- 172.21.15.133:0/322319487 shutdown_connections 2024-02-21T23:26:08.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.863+0000 7f6b0340f640 1 -- 172.21.15.133:0/322319487 wait complete. 2024-02-21T23:26:08.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.865+0000 7f6b0340f640 1 Processor -- start 2024-02-21T23:26:08.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.865+0000 7f6b0340f640 1 -- start start 2024-02-21T23:26:08.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.865+0000 7f6b0340f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc0c6f50 0x7f6afc154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.865+0000 7f6b0340f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6afc0cab80 0x7f6afc0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.865+0000 7f6b0340f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6afc154600 0x7f6afc1f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.865+0000 7f6b0340f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0b5720 con 0x7f6afc154600 2024-02-21T23:26:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.865+0000 7f6b0340f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0cb860 con 0x7f6afc0cab80 2024-02-21T23:26:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.865+0000 7f6b0340f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0b58a0 con 0x7f6afc0c6f50 2024-02-21T23:26:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.866+0000 7f6b011ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc0c6f50 0x7f6afc154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.866+0000 7f6b009aa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6afc0cab80 0x7f6afc0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.866+0000 7f6b011ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc0c6f50 0x7f6afc154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:52812/0 (socket says 172.21.15.133:52812) 2024-02-21T23:26:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.866+0000 7f6b011ab640 1 -- 172.21.15.133:0/1842954268 learned_addr learned my addr 172.21.15.133:0/1842954268 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:08.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.870+0000 7f6b011ab640 1 -- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6afc0cab80 msgr2=0x7f6afc0c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:08.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.870+0000 7f6b011ab640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6afc0cab80 0x7f6afc0c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.870+0000 7f6b011ab640 1 -- 172.21.15.133:0/1842954268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6afc154600 msgr2=0x7f6afc1f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:08.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.870+0000 7f6b011ab640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6afc154600 0x7f6afc1f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.870+0000 7f6b011ab640 1 -- 172.21.15.133:0/1842954268 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc15a040 con 0x7f6afc0c6f50 2024-02-21T23:26:08.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.872+0000 7f6b011ab640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc0c6f50 0x7f6afc154a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6af4000f80 tx=0x7f6af4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.876+0000 7f6af1ffb640 1 -- 172.21.15.133:0/1842954268 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4043020 con 0x7f6afc0c6f50 2024-02-21T23:26:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.876+0000 7f6af1ffb640 1 -- 172.21.15.133:0/1842954268 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af403e450 con 0x7f6afc0c6f50 2024-02-21T23:26:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.876+0000 7f6af1ffb640 1 -- 172.21.15.133:0/1842954268 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4009510 con 0x7f6afc0c6f50 2024-02-21T23:26:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.876+0000 7f6b0340f640 1 -- 172.21.15.133:0/1842954268 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6afc155190 con 0x7f6afc0c6f50 2024-02-21T23:26:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.876+0000 7f6b0340f640 1 -- 172.21.15.133:0/1842954268 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc0c4f50 con 0x7f6afc0c6f50 2024-02-21T23:26:08.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.879+0000 7f6b061ed740 1 Processor -- start 2024-02-21T23:26:08.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.879+0000 7f6b061ed740 1 -- start start 2024-02-21T23:26:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.880+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x5592480e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.880+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592480e4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.880+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592480e2550 0x559248123470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.880+0000 7f6b061ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559248047860 con 0x5592480e2550 2024-02-21T23:26:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.880+0000 7f6b061ed740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559247ff0980 con 0x55924812d970 2024-02-21T23:26:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.880+0000 7f6b061ed740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x559248043aa0 con 0x5592480e3af0 2024-02-21T23:26:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.880+0000 7f6b009aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592480e2550 0x559248123470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.881+0000 7f6b009aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592480e2550 0x559248123470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46192/0 (socket says 172.21.15.133:46192) 2024-02-21T23:26:08.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.881+0000 7f6b009aa640 1 -- 172.21.15.133:0/1935335972 learned_addr learned my addr 172.21.15.133:0/1935335972 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:08.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.881+0000 7f6b011ab640 1 --2- 172.21.15.133:0/1935335972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592480e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.883+0000 7f6b009aa640 1 -- 172.21.15.133:0/1935335972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 msgr2=0x5592480e3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.883+0000 7f6b009aa640 1 --2- 172.21.15.133:0/1935335972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x5592480e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.883+0000 7f6b009aa640 1 -- 172.21.15.133:0/1935335972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 msgr2=0x5592480e4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.883+0000 7f6b009aa640 1 --2- 172.21.15.133:0/1935335972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592480e4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.883+0000 7f6b009aa640 1 -- 172.21.15.133:0/1935335972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559248005c60 con 0x5592480e2550 2024-02-21T23:26:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.885+0000 7f6b009aa640 1 --2- 172.21.15.133:0/1935335972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592480e2550 0x559248123470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009a70 tx=0x7f6aec009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ba199f24e819d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:08.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.888+0000 7f6adeffd640 1 -- 172.21.15.133:0/1935335972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec019070 con 0x5592480e2550 2024-02-21T23:26:08.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.888+0000 7f6adeffd640 1 -- 172.21.15.133:0/1935335972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec004300 con 0x5592480e2550 2024-02-21T23:26:08.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.888+0000 7f6adeffd640 1 -- 172.21.15.133:0/1935335972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec011b60 con 0x5592480e2550 2024-02-21T23:26:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.889+0000 7f6b061ed740 1 -- 172.21.15.133:0/1935335972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592480e2550 msgr2=0x559248123470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.889+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1935335972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592480e2550 0x559248123470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009a70 tx=0x7f6aec009f30 comp rx=0 tx=0).stop 2024-02-21T23:26:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.889+0000 7f6b061ed740 1 -- 172.21.15.133:0/1935335972 shutdown_connections 2024-02-21T23:26:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.889+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1935335972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592480e2550 0x559248123470 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.889+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1935335972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592480e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.889+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1935335972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x5592480e3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.889+0000 7f6b061ed740 1 -- 172.21.15.133:0/1935335972 >> 172.21.15.133:0/1935335972 conn(0x5592480e0260 msgr2=0x5592480e2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.889+0000 7f6b061ed740 1 -- 172.21.15.133:0/1935335972 shutdown_connections 2024-02-21T23:26:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.889+0000 7f6b061ed740 1 -- 172.21.15.133:0/1935335972 wait complete. 2024-02-21T23:26:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.890+0000 7f6b061ed740 1 -- start start 2024-02-21T23:26:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.890+0000 7f6b061ed740 1 Processor -- start 2024-02-21T23:26:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.891+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x559248124ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.891+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592481379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.891+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 0x55924810a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.891+0000 7f6b061ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559248003f10 con 0x559248120a60 2024-02-21T23:26:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.891+0000 7f6b061ed740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924812deb0 con 0x55924812d970 2024-02-21T23:26:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.891+0000 7f6b061ed740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x559248134ec0 con 0x5592480e3af0 2024-02-21T23:26:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.891+0000 7f6b061ed740 10 client.0 _ll_register_callbacks cb 0x559247ff7d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-21T23:26:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.891+0000 7f6b011ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592481379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.892+0000 7f6b011ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592481379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:48224/0 (socket says 172.21.15.133:48224) 2024-02-21T23:26:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.892+0000 7f6b011ab640 1 -- 172.21.15.133:0/2935694463 learned_addr learned my addr 172.21.15.133:0/2935694463 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:08.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.893+0000 7f6b009aa640 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 0x55924810a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.896+0000 7f6b011ab640 1 -- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 msgr2=0x559248124ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.896+0000 7f6b011ab640 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x559248124ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.896+0000 7f6b011ab640 1 -- 172.21.15.133:0/2935694463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 msgr2=0x55924810a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.896+0000 7f6b011ab640 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 0x55924810a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.896+0000 7f6b011ab640 1 -- 172.21.15.133:0/2935694463 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559247d5c2d0 con 0x55924812d970 2024-02-21T23:26:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.897+0000 7f6af1ffb640 1 -- 172.21.15.133:0/1842954268 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6af4046020 con 0x7f6afc0c6f50 2024-02-21T23:26:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.898+0000 7f6af1ffb640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f6ae006b470 0x7f6ae006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.898+0000 7f6b019ac640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f6ae006b470 0x7f6ae006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.898+0000 7f6b019ac640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f6ae006b470 0x7f6ae006d930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009aa0 tx=0x7f6af8006e40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:08.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.899+0000 7f6b011ab640 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592481379f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6af402c4e0 tx=0x7f6af4009000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34ade2b7354828e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:08.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.899+0000 7f6ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-21T23:26:08.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.901+0000 7f6af1ffb640 1 -- 172.21.15.133:0/1842954268 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6af4081650 con 0x7f6afc0c6f50 2024-02-21T23:26:08.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.903+0000 7f6ad7fff640 1 -- 172.21.15.133:0/2935694463 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af407f5f0 con 0x55924812d970 2024-02-21T23:26:08.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.904+0000 7f6ad7fff640 1 -- 172.21.15.133:0/2935694463 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6af40b3180 con 0x55924812d970 2024-02-21T23:26:08.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.904+0000 7f6b061ed740 1 -- 172.21.15.133:0/2935694463 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6aec004300 con 0x55924812d970 2024-02-21T23:26:08.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.905+0000 7f6ad7fff640 1 -- 172.21.15.133:0/2935694463 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af4082cb0 con 0x55924812d970 2024-02-21T23:26:08.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.905+0000 7f6b061ed740 10 client.4768 fetch_fsmap learned FSMap version 10 2024-02-21T23:26:08.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.905+0000 7f6b061ed740 1 -- 172.21.15.133:0/2935694463 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0x559247e266b0 con 0x55924812d970 2024-02-21T23:26:08.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.948+0000 7f6ad7fff640 1 -- 172.21.15.133:0/2935694463 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec004300 con 0x55924812d970 2024-02-21T23:26:08.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.948+0000 7f6ad7fff640 1 -- 172.21.15.133:0/2935694463 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 10) v1 ==== 4452+0+0 (secure 0 0 0) 0x7f6af407f9c0 con 0x55924812d970 2024-02-21T23:26:08.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.948+0000 7f6b061ed740 10 client.4768 fetch_fsmap finished waiting for FSMap version 10 2024-02-21T23:26:08.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.948+0000 7f6b061ed740 10 client.4768 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-21T23:26:08.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.948+0000 7f6b061ed740 20 client.4768 populate_metadata read hostname 'smithi133' 2024-02-21T23:26:08.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.948+0000 7f6b061ed740 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x559248141060 0x559248143450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.948+0000 7f6b061ed740 4 client.4768 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-21T23:26:08.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.948+0000 7f6b061ed740 1 -- 172.21.15.133:0/2935694463 --> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592480a5480 con 0x559248141060 2024-02-21T23:26:08.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.949+0000 7f6b009aa640 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x559248141060 0x559248143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.952+0000 7f6b009aa640 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x559248141060 0x559248143450 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0c8decde440a27 server_cookie=f7c232d3fe28fd8e in_seq=0 out_seq=0 2024-02-21T23:26:08.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.952+0000 7f6ad7fff640 10 client.4768 ms_handle_connect on v2:172.21.15.142:6834/1527124199 2024-02-21T23:26:08.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6ad7fff640 1 -- 172.21.15.133:0/2935694463 <== mds.0 v2:172.21.15.142:6834/1527124199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5592480a5480 con 0x559248141060 2024-02-21T23:26:08.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6ad7fff640 10 client.4768 handle_command_reply: tid=0 2024-02-21T23:26:08.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 client.4768 shutdown 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 -- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 msgr2=0x5592481379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592481379f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6af402c4e0 tx=0x7f6af4009000 comp rx=0 tx=0).stop 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 -- 172.21.15.133:0/2935694463 shutdown_connections 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x559248141060 0x559248143450 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 0x55924810a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6ad7fff640 0 client.4768 ms_handle_reset on v2:172.21.15.142:6834/1527124199 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x5592481379f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 --2- 172.21.15.133:0/2935694463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x559248124ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 -- 172.21.15.133:0/2935694463 >> 172.21.15.133:0/2935694463 conn(0x5592480e2550 msgr2=0x5592480dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 -- 172.21.15.133:0/2935694463 shutdown_connections 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.971+0000 7f6b061ed740 1 -- 172.21.15.133:0/2935694463 wait complete. 2024-02-21T23:26:08.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.972+0000 7f6b061ed740 20 client.4768 trim_cache size 0 max 16384 2024-02-21T23:26:08.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.989+0000 7f6b061ed740 1 Processor -- start 2024-02-21T23:26:08.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.989+0000 7f6b061ed740 1 -- start start 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5592480e3af0 0x5592481375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559248120a60 0x559248137b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924812d970 0x55924810a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b061ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc1fced0 con 0x55924812d970 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b061ed740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559248003f10 con 0x5592480e3af0 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b061ed740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55924812deb0 con 0x559248120a60 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b019ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924812d970 0x55924810a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b009aa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559248120a60 0x559248137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b011ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5592480e3af0 0x5592481375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b009aa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559248120a60 0x559248137b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:52842/0 (socket says 172.21.15.133:52842) 2024-02-21T23:26:08.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.990+0000 7f6b009aa640 1 -- 172.21.15.133:0/449556315 learned_addr learned my addr 172.21.15.133:0/449556315 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:08.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.993+0000 7f6b019ac640 1 -- 172.21.15.133:0/449556315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559248120a60 msgr2=0x559248137b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.993+0000 7f6b019ac640 1 --2- 172.21.15.133:0/449556315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559248120a60 0x559248137b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.993+0000 7f6b019ac640 1 -- 172.21.15.133:0/449556315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5592480e3af0 msgr2=0x5592481375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.993+0000 7f6b019ac640 1 --2- 172.21.15.133:0/449556315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5592480e3af0 0x5592481375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.993+0000 7f6b019ac640 1 -- 172.21.15.133:0/449556315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559248122290 con 0x55924812d970 2024-02-21T23:26:08.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.993+0000 7f6b009aa640 1 --2- 172.21.15.133:0/449556315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559248120a60 0x559248137b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:26:08.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.994+0000 7f6b019ac640 1 --2- 172.21.15.133:0/449556315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924812d970 0x55924810a310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009560 tx=0x7f6af8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6718409eb6157b4f server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:08.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.998+0000 7f6ade7fc640 1 -- 172.21.15.133:0/449556315 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8025070 con 0x55924812d970 2024-02-21T23:26:08.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.998+0000 7f6ade7fc640 1 -- 172.21.15.133:0/449556315 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af800eca0 con 0x55924812d970 2024-02-21T23:26:08.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.998+0000 7f6ade7fc640 1 -- 172.21.15.133:0/449556315 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8003d80 con 0x55924812d970 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 -- 172.21.15.133:0/449556315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924812d970 msgr2=0x55924810a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 --2- 172.21.15.133:0/449556315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924812d970 0x55924810a310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009560 tx=0x7f6af8011000 comp rx=0 tx=0).stop 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 -- 172.21.15.133:0/449556315 shutdown_connections 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 --2- 172.21.15.133:0/449556315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924812d970 0x55924810a310 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 --2- 172.21.15.133:0/449556315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559248120a60 0x559248137b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 --2- 172.21.15.133:0/449556315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5592480e3af0 0x5592481375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 -- 172.21.15.133:0/449556315 >> 172.21.15.133:0/449556315 conn(0x5592480e2550 msgr2=0x55924811fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 -- 172.21.15.133:0/449556315 shutdown_connections 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 -- 172.21.15.133:0/449556315 wait complete. 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 -- start start 2024-02-21T23:26:08.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:08.999+0000 7f6b061ed740 1 Processor -- start 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.000+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x5592481379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.000+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 0x5592481243e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.000+0000 7f6b061ed740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x55924818de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.000+0000 7f6b061ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924810bff0 con 0x559248120a60 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.000+0000 7f6b061ed740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924810b400 con 0x55924812d970 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.000+0000 7f6b061ed740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55924811eab0 con 0x5592480e3af0 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.000+0000 7f6b061ed740 10 client.0 _ll_register_callbacks cb 0x7f6af40071f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.001+0000 7f6b009aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 0x5592481243e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.001+0000 7f6b011ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x5592481379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.001+0000 7f6b019ac640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x55924818de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.001+0000 7f6b011ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x5592481379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:52856/0 (socket says 172.21.15.133:52856) 2024-02-21T23:26:08.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.001+0000 7f6b011ab640 1 -- 172.21.15.133:0/1451854775 learned_addr learned my addr 172.21.15.133:0/1451854775 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:08.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.004+0000 7f6b011ab640 1 -- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 msgr2=0x55924818de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.004+0000 7f6b011ab640 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x55924818de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.004+0000 7f6b011ab640 1 -- 172.21.15.133:0/1451854775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 msgr2=0x5592481243e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:26:08.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.004+0000 7f6b011ab640 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 0x5592481243e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:08.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.004+0000 7f6b011ab640 1 -- 172.21.15.133:0/1451854775 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559248148c70 con 0x5592480e3af0 2024-02-21T23:26:08.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.004+0000 7f6b009aa640 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 0x5592481243e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:26:08.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.006+0000 7f6b011ab640 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x5592481379f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6af4087660 tx=0x7f6af40b7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a0c9faadd5bfafc server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:08.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.006+0000 7f6ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3301/0 2024-02-21T23:26:08.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.008+0000 7f6ad7fff640 1 -- 172.21.15.133:0/1451854775 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4082e90 con 0x5592480e3af0 2024-02-21T23:26:08.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.008+0000 7f6b061ed740 1 -- 172.21.15.133:0/1451854775 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6af800eca0 con 0x5592480e3af0 2024-02-21T23:26:08.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.011+0000 7f6ad7fff640 1 -- 172.21.15.133:0/1451854775 <== mon.2 v2:172.21.15.142:3301/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6af407f9c0 con 0x5592480e3af0 2024-02-21T23:26:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.011+0000 7f6ad7fff640 1 -- 172.21.15.133:0/1451854775 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af4083120 con 0x5592480e3af0 2024-02-21T23:26:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.011+0000 7f6b061ed740 10 client.4601 fetch_fsmap learned FSMap version 10 2024-02-21T23:26:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.011+0000 7f6b061ed740 1 -- 172.21.15.133:0/1451854775 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0x55924812d780 con 0x5592480e3af0 2024-02-21T23:26:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.023+0000 7f6ad7fff640 1 -- 172.21.15.133:0/1451854775 <== mon.2 v2:172.21.15.142:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af800eca0 con 0x5592480e3af0 2024-02-21T23:26:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.024+0000 7f6ad7fff640 1 -- 172.21.15.133:0/1451854775 <== mon.2 v2:172.21.15.142:3301/0 5 ==== fsmap(e 10) v1 ==== 4452+0+0 (secure 0 0 0) 0x7f6af40a5030 con 0x5592480e3af0 2024-02-21T23:26:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.024+0000 7f6b061ed740 10 client.4601 fetch_fsmap finished waiting for FSMap version 10 2024-02-21T23:26:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.024+0000 7f6b061ed740 10 client.4601 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-21T23:26:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.024+0000 7f6b061ed740 20 client.4601 populate_metadata read hostname 'smithi133' 2024-02-21T23:26:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.024+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x559248141060 0x559248147820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.024+0000 7f6b061ed740 4 client.4601 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-21T23:26:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.024+0000 7f6b061ed740 1 -- 172.21.15.133:0/1451854775 --> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559248054da0 con 0x559248141060 2024-02-21T23:26:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.025+0000 7f6b009aa640 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x559248141060 0x559248147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:09.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.028+0000 7f6b009aa640 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x559248141060 0x559248147820 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=384ccd6e135368d8 server_cookie=e898337b1e5a9346 in_seq=0 out_seq=0 2024-02-21T23:26:09.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.028+0000 7f6ad7fff640 10 client.4601 ms_handle_connect on v2:172.21.15.142:6834/1527124199 2024-02-21T23:26:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.031+0000 7f6ad7fff640 1 -- 172.21.15.133:0/1451854775 <== mds.0 v2:172.21.15.142:6834/1527124199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x559248054da0 con 0x559248141060 2024-02-21T23:26:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.031+0000 7f6ad7fff640 10 client.4601 handle_command_reply: tid=0 2024-02-21T23:26:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 client.4601 shutdown 2024-02-21T23:26:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 -- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 msgr2=0x5592481379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x5592481379f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6af4087660 tx=0x7f6af40b7000 comp rx=0 tx=0).stop 2024-02-21T23:26:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 -- 172.21.15.133:0/1451854775 shutdown_connections 2024-02-21T23:26:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55924812d970 0x55924818de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:6834/1527124199,v1:172.21.15.142:6835/1527124199] conn(0x559248141060 0x559248147820 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559248120a60 0x5592481243e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 --2- 172.21.15.133:0/1451854775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5592480e3af0 0x5592481379f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6ad7fff640 0 client.4601 ms_handle_reset on v2:172.21.15.142:6834/1527124199 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 -- 172.21.15.133:0/1451854775 >> 172.21.15.133:0/1451854775 conn(0x5592480e2550 msgr2=0x5592480e2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 -- 172.21.15.133:0/1451854775 shutdown_connections 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 1 -- 172.21.15.133:0/1451854775 wait complete. 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.032+0000 7f6b061ed740 20 client.4601 trim_cache size 0 max 16384 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "1f1c3b6d-6263-48c3-85b7-ccc808770c4d", 2024-02-21T23:26:09.020 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4269, 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 51.658269885999999, 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 10, 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 26, 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 24, 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 59.011829669999997 2024-02-21T23:26:09.021 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 -- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f6ae006b470 msgr2=0x7f6ae006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f6ae006b470 0x7f6ae006d930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009aa0 tx=0x7f6af8006e40 comp rx=0 tx=0).stop 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 -- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc0c6f50 msgr2=0x7f6afc154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc0c6f50 0x7f6afc154a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6af4000f80 tx=0x7f6af4004a40 comp rx=0 tx=0).stop 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 -- 172.21.15.133:0/1842954268 shutdown_connections 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f6ae006b470 0x7f6ae006d930 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6afc154600 0x7f6afc1f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6afc0cab80 0x7f6afc0c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 --2- 172.21.15.133:0/1842954268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6afc0c6f50 0x7f6afc154a30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 -- 172.21.15.133:0/1842954268 >> 172.21.15.133:0/1842954268 conn(0x7f6afc062000 msgr2=0x7f6afc14e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:09.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 -- 172.21.15.133:0/1842954268 shutdown_connections 2024-02-21T23:26:09.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.033+0000 7f6b0340f640 1 -- 172.21.15.133:0/1842954268 wait complete. 2024-02-21T23:26:09.031 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-02-21T23:26:09.041 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-02-21T23:26:09.041 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-02-21T23:26:09.041 INFO:tasks.ceph_fuse:client.0 config is {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']} 2024-02-21T23:26:09.041 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-21T23:26:09.041 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:09.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:09.042 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-02-21T23:26:09.056 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-02-21T23:26:09.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:09.057 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-21T23:26:09.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:09.124 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-02-21T23:26:09.217 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-02-21T23:26:09.217 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-21T23:26:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.331+0000 7fa86ec20640 1 Processor -- start 2024-02-21T23:26:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.331+0000 7fa86ec20640 1 -- start start 2024-02-21T23:26:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.332+0000 7fa86ec20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa86815bec0 0x7fa868071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.332+0000 7fa86ec20640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa868158230 con 0x7fa8680c53a0 2024-02-21T23:26:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.332+0000 7fa86ec20640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa868158510 con 0x7fa8680c4fc0 2024-02-21T23:26:09.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.332+0000 7fa86ec20640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa868158eb0 con 0x7fa86815bec0 2024-02-21T23:26:09.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.333+0000 7fa867fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa8680c53a0 0x7fa86815b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45214/0 (socket says 172.21.15.133:45214) 2024-02-21T23:26:09.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.333+0000 7fa867fff640 1 -- 172.21.15.133:0/1038210483 learned_addr learned my addr 172.21.15.133:0/1038210483 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:26:09.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.333+0000 7fa86d1bd640 1 --2- 172.21.15.133:0/1038210483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa86815bec0 0x7fa868071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:09.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.335+0000 7fa8677fe640 1 -- 172.21.15.133:0/1038210483 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984132851 0 0) 0x7fa868158230 con 0x7fa8680c53a0 2024-02-21T23:26:09.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.335+0000 7fa8677fe640 1 -- 172.21.15.133:0/1038210483 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa854003520 con 0x7fa8680c53a0 2024-02-21T23:26:09.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.336+0000 7fa8677fe640 1 -- 172.21.15.133:0/1038210483 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284691010 0 0) 0x7fa868158510 con 0x7fa8680c4fc0 2024-02-21T23:26:09.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.336+0000 7fa8677fe640 1 -- 172.21.15.133:0/1038210483 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa868158230 con 0x7fa8680c4fc0 2024-02-21T23:26:09.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.336+0000 7fa86d1bd640 1 -- 172.21.15.133:0/1038210483 >> v1:172.21.15.142:6789/0 conn(0x7fa8680c4fc0 legacy=0x7fa86806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.336+0000 7fa86d1bd640 1 -- 172.21.15.133:0/1038210483 >> v1:172.21.15.133:6789/0 conn(0x7fa8680c53a0 legacy=0x7fa86815b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.336+0000 7fa86d1bd640 1 -- 172.21.15.133:0/1038210483 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8681587f0 con 0x7fa86815bec0 2024-02-21T23:26:09.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.338+0000 7fa86d1bd640 1 --2- 172.21.15.133:0/1038210483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa86815bec0 0x7fa868071b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa8600097d0 tx=0x7fa86002b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a694c318498cc8 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:09.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.342+0000 7fa8677fe640 1 -- 172.21.15.133:0/1038210483 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860034070 con 0x7fa86815bec0 2024-02-21T23:26:09.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.342+0000 7fa8677fe640 1 -- 172.21.15.133:0/1038210483 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860030880 con 0x7fa86815bec0 2024-02-21T23:26:09.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.342+0000 7fa8677fe640 1 -- 172.21.15.133:0/1038210483 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860030c50 con 0x7fa86815bec0 2024-02-21T23:26:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.343+0000 7fa86ec20640 1 -- 172.21.15.133:0/1038210483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa86815bec0 msgr2=0x7fa868071b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.343+0000 7fa86ec20640 1 --2- 172.21.15.133:0/1038210483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa86815bec0 0x7fa868071b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa8600097d0 tx=0x7fa86002b9c0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.343+0000 7fa86ec20640 1 -- 172.21.15.133:0/1038210483 shutdown_connections 2024-02-21T23:26:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.343+0000 7fa86ec20640 1 --2- 172.21.15.133:0/1038210483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa86815bec0 0x7fa868071b60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.343+0000 7fa86ec20640 1 -- 172.21.15.133:0/1038210483 >> 172.21.15.133:0/1038210483 conn(0x7fa86806aec0 msgr2=0x7fa86806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.343+0000 7fa86ec20640 1 -- 172.21.15.133:0/1038210483 shutdown_connections 2024-02-21T23:26:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.343+0000 7fa86ec20640 1 -- 172.21.15.133:0/1038210483 wait complete. 2024-02-21T23:26:09.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.344+0000 7fa86ec20640 1 Processor -- start 2024-02-21T23:26:09.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.344+0000 7fa86ec20640 1 -- start start 2024-02-21T23:26:09.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.345+0000 7fa86ec20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8680c4fc0 0x7fa8681ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:09.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.345+0000 7fa86ec20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8680c53a0 0x7fa868143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:09.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.345+0000 7fa86ec20640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa86815bec0 0x7fa868144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:09.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.345+0000 7fa86ec20640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86806f8d0 con 0x7fa8680c4fc0 2024-02-21T23:26:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.345+0000 7fa86ec20640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa868157f20 con 0x7fa86815bec0 2024-02-21T23:26:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.345+0000 7fa86ec20640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86806fa50 con 0x7fa8680c53a0 2024-02-21T23:26:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.345+0000 7fa86c9bc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8680c4fc0 0x7fa8681ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.346+0000 7fa86d1bd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa86815bec0 0x7fa868144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.346+0000 7fa867fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8680c53a0 0x7fa868143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.346+0000 7fa867fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8680c53a0 0x7fa868143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:52880/0 (socket says 172.21.15.133:52880) 2024-02-21T23:26:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.346+0000 7fa867fff640 1 -- 172.21.15.133:0/3678699992 learned_addr learned my addr 172.21.15.133:0/3678699992 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:26:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.349+0000 7fa86c9bc640 1 -- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8680c53a0 msgr2=0x7fa868143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.349+0000 7fa86c9bc640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8680c53a0 0x7fa868143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.349+0000 7fa86c9bc640 1 -- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa86815bec0 msgr2=0x7fa868144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.349+0000 7fa86c9bc640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa86815bec0 0x7fa868144430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.349+0000 7fa86c9bc640 1 -- 172.21.15.133:0/3678699992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8680cafe0 con 0x7fa8680c4fc0 2024-02-21T23:26:09.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.351+0000 7fa86c9bc640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8680c4fc0 0x7fa8681ee0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa85000c3c0 tx=0x7fa85000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.355+0000 7fa8657fa640 1 -- 172.21.15.133:0/3678699992 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850017070 con 0x7fa8680c4fc0 2024-02-21T23:26:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.355+0000 7fa8657fa640 1 -- 172.21.15.133:0/3678699992 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8500052f0 con 0x7fa8680c4fc0 2024-02-21T23:26:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.355+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8681ed8c0 con 0x7fa8680c4fc0 2024-02-21T23:26:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.355+0000 7fa8657fa640 1 -- 172.21.15.133:0/3678699992 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850003830 con 0x7fa8680c4fc0 2024-02-21T23:26:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.355+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8680c95c0 con 0x7fa8680c4fc0 2024-02-21T23:26:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.356+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86806a130 con 0x7fa8680c4fc0 2024-02-21T23:26:09.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.374+0000 7fa8657fa640 1 -- 172.21.15.133:0/3678699992 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa85001a070 con 0x7fa8680c4fc0 2024-02-21T23:26:09.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.374+0000 7fa8657fa640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa84806b4d0 0x7fa84806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:26:09.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.375+0000 7fa867fff640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa84806b4d0 0x7fa84806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:26:09.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.375+0000 7fa867fff640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa84806b4d0 0x7fa84806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa86814dfd0 tx=0x7fa858009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:26:09.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.612+0000 7fa8657fa640 1 -- 172.21.15.133:0/3678699992 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fa8500562c0 con 0x7fa8680c4fc0 2024-02-21T23:26:09.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.612+0000 7fa8657fa640 1 -- 172.21.15.133:0/3678699992 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fa850021020 con 0x7fa8680c4fc0 2024-02-21T23:26:09.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.825+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa8680c1a70 con 0x7fa8680c4fc0 2024-02-21T23:26:09.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.831+0000 7fa8657fa640 1 -- 172.21.15.133:0/3678699992 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v10) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa8680cafe0 con 0x7fa8680c4fc0 2024-02-21T23:26:09.817 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-21T23:26:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa84806b4d0 msgr2=0x7fa84806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa84806b4d0 0x7fa84806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa86814dfd0 tx=0x7fa858009040 comp rx=0 tx=0).stop 2024-02-21T23:26:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8680c4fc0 msgr2=0x7fa8681ee0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:26:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8680c4fc0 0x7fa8681ee0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa85000c3c0 tx=0x7fa85000c880 comp rx=0 tx=0).stop 2024-02-21T23:26:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 shutdown_connections 2024-02-21T23:26:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa84806b4d0 0x7fa84806d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa86815bec0 0x7fa868144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8680c53a0 0x7fa868143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 --2- 172.21.15.133:0/3678699992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8680c4fc0 0x7fa8681ee0a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:26:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 >> 172.21.15.133:0/3678699992 conn(0x7fa86806aec0 msgr2=0x7fa8680c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:26:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 shutdown_connections 2024-02-21T23:26:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:09.833+0000 7fa86ec20640 1 -- 172.21.15.133:0/3678699992 wait complete. 2024-02-21T23:26:09.831 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-21T23:26:09.831 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-21T23:26:09.831 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-02-21T23:26:09.831 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-21T23:26:09.831 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:09.831 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-21T23:26:09.831 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-21T23:26:09.831 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-21T23:26:09.831 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-21T23:26:09.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:09.832 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-02-21T23:26:09.847 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-21T23:26:09.847 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-21T23:26:09.847 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-02-21T23:26:09.847 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-02-21T23:26:09.847 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-02-21T23:26:09.847 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-02-21T23:26:09.847 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-21T23:26:09.847 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41930sec preferred_lft 41930sec 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-02-21T23:26:09.848 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-02-21T23:26:09.849 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-21T23:26:09.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:09.849 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:26:09.849 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:26:09.849 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2024-02-21T23:26:09.849 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2024-02-21T23:26:09.849 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2024-02-21T23:26:09.849 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-21T23:26:09.849 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:26:09.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:09.955 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-21T23:26:09.981 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-02-21T23:26:09.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:09.982 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-02-21T23:26:10.038 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-02-21T23:26:10.038 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-21T23:26:10.038 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-21T23:26:10.038 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-21T23:26:10.038 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-21T23:26:10.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:10.039 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:26:10.039 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:26:10.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-21T23:26:10.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-21T23:26:10.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-21T23:26:10.039 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:26:10.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:10.135 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-02-21T23:26:10.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:10.149 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-02-21T23:26:10.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:10.204 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:26:10.205 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:26:10.205 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:10.205 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-21T23:26:10.205 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:26:10.287 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-21T23:26:10.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:10.287 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:26:10.287 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:26:10.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-21T23:26:10.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-21T23:26:10.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-21T23:26:10.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-21T23:26:10.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-21T23:26:10.288 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:26:10.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:10.413 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-02-21T23:26:10.413 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-02-21T23:26:10.413 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-02-21T23:26:10.413 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2024-02-21T23:26:10.413 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-02-21T23:26:10.454 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']} 2024-02-21T23:26:10.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-21T23:26:10.455 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:10.509 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-21T23:26:10.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-21T23:26:10.509 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:10.564 DEBUG:tasks.ceph_manager:running client.0 under valgrind with args ['sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/client.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-02-21T23:26:10.564 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:26:10.633 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:26:10.688 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:26:10.688 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:26:10.688 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:26:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:26:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:26:10.691 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:26:10.691 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:26:10.691 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:26:10.691 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:10.691 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:10.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:10.691 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:26:10.743 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-21T23:26:10.743 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/client.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-02-21T23:26:10.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:26:10.812 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:26:10.869 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:26:10.869 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:26:10.869 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:26:10.869 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:26:10.869 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:26:10.870 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:10.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:10.872 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:26:12.025 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:26:12.054 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:26:12.110 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:26:12.110 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:26:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:26:12.113 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:26:12.113 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:26:12.113 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:26:12.113 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:12.113 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:12.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:12.113 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:26:13.267 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:26:13.295 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:26:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:26:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:26:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:26:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:26:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:26:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:26:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:26:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:26:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:26:13.352 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:13.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:13.353 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:26:14.507 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:26:14.534 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:26:14.590 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:26:14.590 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:26:14.590 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:26:14.590 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:26:14.590 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:26:14.590 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:26:14.591 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:14.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:14.593 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:26:15.747 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:26:15.776 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:26:15.832 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:26:15.832 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:26:15.832 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:26:15.832 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:26:15.832 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:26:15.832 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:26:15.833 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:15.834 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:15.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:15.835 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:26:16.522 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[46098]: starting ceph client2024-02-21T23:26:16.427+0000 650c100 -1 init, newargv = 0x7c18f90 newargc=15 2024-02-21T23:26:16.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr: 2024-02-21T23:26:16.989 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:26:17.017 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:26:17.073 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:26:17.073 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:26:17.073 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:26:17.073 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:26:17.073 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:26:17.073 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:26:17.073 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:26:17.073 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:26:17.074 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:17.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:17.075 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:26:17.183 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[46098]: starting fuse 2024-02-21T23:26:18.230 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-02-21T23:26:18.259 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-02-21T23:26:18.315 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-02-21T23:26:18.315 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-02-21T23:26:18.315 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-02-21T23:26:18.315 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-02-21T23:26:18.315 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-02-21T23:26:18.315 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/selinux 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-02-21T23:26:18.316 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysctl.service 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/nfs/rpc_pipefs 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:26:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:18.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:18.318 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-02-21T23:26:18.370 INFO:teuthology.orchestra.run.smithi133.stdout:41 2024-02-21T23:26:18.370 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-02-21T23:26:18.371 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-02-21T23:26:18.372 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-02-21T23:26:18.491 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.46098.asok 2024-02-21T23:26:18.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:26:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-21T23:26:18.502 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46098.asok 2024-02-21T23:26:18.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:18.502 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46098.asok status 2024-02-21T23:26:18.739 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "3bfc837d7b45ce8e095d92fa9c342964cea5e95f", 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1643-g3bfc837d (3bfc837d7b45ce8e095d92fa9c342964cea5e95f) squid (dev)", 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "46098", 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-02-21T23:26:18.740 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4613, 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 4613 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-02-21T23:26:18.741 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3324036237 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3324036237 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.4613 172.21.15.133:0/3324036237", 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "172.21.15.133:0/3324036237", 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 10, 2024-02-21T23:26:18.742 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 26, 2024-02-21T23:26:18.743 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-02-21T23:26:18.743 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-02-21T23:26:18.743 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-02-21T23:26:18.743 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:26:18.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:18.752 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-02-21T23:26:18.753 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:18.802 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-02-21T23:26:18.803 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:18.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:26:18.803 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:18.856 INFO:teuthology.run_tasks:Running task workunit... 2024-02-21T23:26:18.869 INFO:tasks.workunit:Pulling workunits from ref 3bfc837d7b45ce8e095d92fa9c342964cea5e95f 2024-02-21T23:26:18.870 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-21T23:26:18.870 INFO:tasks.workunit:timeout=3h 2024-02-21T23:26:18.870 INFO:tasks.workunit:cleanup=True 2024-02-21T23:26:18.870 DEBUG:teuthology.orchestra.run.smithi133:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:18.886 INFO:teuthology.orchestra.run.smithi133.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:18.886 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-21T23:26:18.886 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 29h/41d Inode: 1 Links: 2 2024-02-21T23:26:18.886 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-21T23:26:18.886 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-21T23:26:18.886 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-21T23:26:18.886 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-02-21 23:25:17.681903091 +0000 2024-02-21T23:26:18.886 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-02-21 23:25:30.767151088 +0000 2024-02-21T23:26:18.886 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-02-21T23:26:18.886 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-21T23:26:18.887 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-21T23:26:19.280 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3bfc837d7b45ce8e095d92fa9c342964cea5e95f 2024-02-21T23:26:19.295 INFO:tasks.workunit.client.0.smithi133.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-21T23:26:26.428 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:26:26.432 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:26:56.570 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:26:56.572 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:27:26.664 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:27:26.668 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:27:32.636 INFO:tasks.workunit.client.0.smithi133.stderr:Note: switching to '3bfc837d7b45ce8e095d92fa9c342964cea5e95f'. 2024-02-21T23:27:32.636 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-02-21T23:27:32.636 INFO:tasks.workunit.client.0.smithi133.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-21T23:27:32.636 INFO:tasks.workunit.client.0.smithi133.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-21T23:27:32.636 INFO:tasks.workunit.client.0.smithi133.stderr:state without impacting any branches by switching back to a branch. 2024-02-21T23:27:32.636 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-02-21T23:27:32.636 INFO:tasks.workunit.client.0.smithi133.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-21T23:27:32.636 INFO:tasks.workunit.client.0.smithi133.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-21T23:27:32.636 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-02-21T23:27:32.637 INFO:tasks.workunit.client.0.smithi133.stderr: git switch -c 2024-02-21T23:27:32.637 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-02-21T23:27:32.637 INFO:tasks.workunit.client.0.smithi133.stderr:Or undo this operation with: 2024-02-21T23:27:32.637 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-02-21T23:27:32.637 INFO:tasks.workunit.client.0.smithi133.stderr: git switch - 2024-02-21T23:27:32.637 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-02-21T23:27:32.637 INFO:tasks.workunit.client.0.smithi133.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-21T23:27:32.637 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-02-21T23:27:32.637 INFO:tasks.workunit.client.0.smithi133.stderr:HEAD is now at 3bfc837d7b4 test workaround 2024-02-21T23:27:32.639 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-21T23:27:32.678 INFO:tasks.workunit.client.0.smithi133.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-21T23:27:32.681 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-21T23:27:32.681 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-21T23:27:33.377 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-21T23:27:33.442 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-21T23:27:33.493 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-21T23:27:33.494 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-21T23:27:33.494 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-21T23:27:33.534 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-21T23:27:33.537 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:27:33.537 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-21T23:27:33.593 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-02-21T23:27:33.594 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-02-21T23:27:33.594 DEBUG:teuthology.orchestra.run.smithi133:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3bfc837d7b45ce8e095d92fa9c342964cea5e95f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-02-21T23:27:33.827 INFO:tasks.workunit.client.0.smithi133.stderr:+ dbench 1 2024-02-21T23:27:33.844 INFO:tasks.workunit.client.0.smithi133.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-02-21T23:27:33.844 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-02-21T23:27:33.844 INFO:tasks.workunit.client.0.smithi133.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-02-21T23:27:33.855 INFO:tasks.workunit.client.0.smithi133.stdout:0 of 1 processes prepared for launch 0 sec 2024-02-21T23:27:33.953 INFO:tasks.workunit.client.0.smithi133.stdout:1 of 1 processes prepared for launch 0 sec 2024-02-21T23:27:33.953 INFO:tasks.workunit.client.0.smithi133.stdout:releasing clients 2024-02-21T23:27:34.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 31 1.05 MB/sec warmup 1 sec latency 42.539 ms 2024-02-21T23:27:35.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 113 2.62 MB/sec warmup 2 sec latency 200.208 ms 2024-02-21T23:27:36.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 155 2.18 MB/sec warmup 3 sec latency 175.478 ms 2024-02-21T23:27:37.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 199 1.96 MB/sec warmup 4 sec latency 172.419 ms 2024-02-21T23:27:38.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 247 1.88 MB/sec warmup 5 sec latency 162.511 ms 2024-02-21T23:27:39.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 289 1.78 MB/sec warmup 6 sec latency 186.563 ms 2024-02-21T23:27:40.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 331 1.71 MB/sec warmup 7 sec latency 246.138 ms 2024-02-21T23:27:41.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 373 1.66 MB/sec warmup 8 sec latency 210.026 ms 2024-02-21T23:27:42.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 413 1.62 MB/sec warmup 9 sec latency 199.640 ms 2024-02-21T23:27:43.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 446 1.56 MB/sec warmup 10 sec latency 264.679 ms 2024-02-21T23:27:44.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 485 1.54 MB/sec warmup 11 sec latency 263.641 ms 2024-02-21T23:27:45.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 521 1.50 MB/sec warmup 12 sec latency 318.908 ms 2024-02-21T23:27:46.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 608 1.61 MB/sec warmup 13 sec latency 241.798 ms 2024-02-21T23:27:47.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 644 1.50 MB/sec warmup 14 sec latency 355.115 ms 2024-02-21T23:27:48.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 653 1.40 MB/sec warmup 15 sec latency 448.057 ms 2024-02-21T23:27:49.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 676 1.32 MB/sec warmup 16 sec latency 448.769 ms 2024-02-21T23:27:50.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 694 1.25 MB/sec warmup 17 sec latency 536.974 ms 2024-02-21T23:27:51.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 728 1.18 MB/sec warmup 18 sec latency 308.097 ms 2024-02-21T23:27:52.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 746 1.12 MB/sec warmup 19 sec latency 324.541 ms 2024-02-21T23:27:53.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 789 1.07 MB/sec warmup 20 sec latency 341.116 ms 2024-02-21T23:27:54.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 813 1.02 MB/sec warmup 21 sec latency 335.015 ms 2024-02-21T23:27:55.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 823 0.98 MB/sec warmup 22 sec latency 405.739 ms 2024-02-21T23:27:56.878 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:27:56.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:27:56.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 861 0.94 MB/sec warmup 23 sec latency 337.678 ms 2024-02-21T23:27:57.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 880 0.90 MB/sec warmup 24 sec latency 319.618 ms 2024-02-21T23:27:58.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 913 0.87 MB/sec warmup 25 sec latency 319.112 ms 2024-02-21T23:27:59.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 929 0.84 MB/sec warmup 26 sec latency 311.471 ms 2024-02-21T23:28:00.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 937 0.81 MB/sec warmup 27 sec latency 356.807 ms 2024-02-21T23:28:01.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 970 0.78 MB/sec warmup 28 sec latency 323.378 ms 2024-02-21T23:28:02.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1000 0.76 MB/sec warmup 29 sec latency 318.887 ms 2024-02-21T23:28:03.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1036 0.76 MB/sec warmup 30 sec latency 308.386 ms 2024-02-21T23:28:04.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1048 0.74 MB/sec warmup 31 sec latency 320.153 ms 2024-02-21T23:28:05.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1067 0.72 MB/sec warmup 32 sec latency 328.777 ms 2024-02-21T23:28:06.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1100 0.70 MB/sec warmup 33 sec latency 322.551 ms 2024-02-21T23:28:07.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1128 0.68 MB/sec warmup 34 sec latency 311.925 ms 2024-02-21T23:28:08.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1157 0.66 MB/sec warmup 35 sec latency 246.222 ms 2024-02-21T23:28:09.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1170 0.64 MB/sec warmup 36 sec latency 334.272 ms 2024-02-21T23:28:10.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1181 0.62 MB/sec warmup 37 sec latency 303.237 ms 2024-02-21T23:28:11.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1214 0.61 MB/sec warmup 38 sec latency 380.092 ms 2024-02-21T23:28:12.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1248 0.59 MB/sec warmup 39 sec latency 213.466 ms 2024-02-21T23:28:13.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1278 0.58 MB/sec warmup 40 sec latency 338.268 ms 2024-02-21T23:28:14.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1284 0.56 MB/sec warmup 41 sec latency 336.925 ms 2024-02-21T23:28:15.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1377 0.62 MB/sec warmup 42 sec latency 352.547 ms 2024-02-21T23:28:16.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1387 0.60 MB/sec warmup 43 sec latency 321.292 ms 2024-02-21T23:28:17.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1413 0.59 MB/sec warmup 44 sec latency 321.269 ms 2024-02-21T23:28:18.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1445 0.58 MB/sec warmup 45 sec latency 315.007 ms 2024-02-21T23:28:19.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1465 0.57 MB/sec warmup 46 sec latency 339.715 ms 2024-02-21T23:28:20.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1487 0.55 MB/sec warmup 47 sec latency 305.779 ms 2024-02-21T23:28:21.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1509 0.54 MB/sec warmup 48 sec latency 336.457 ms 2024-02-21T23:28:22.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1530 0.53 MB/sec warmup 49 sec latency 313.856 ms 2024-02-21T23:28:23.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1556 0.52 MB/sec warmup 50 sec latency 297.986 ms 2024-02-21T23:28:24.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1589 0.51 MB/sec warmup 51 sec latency 94.235 ms 2024-02-21T23:28:25.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1610 0.50 MB/sec warmup 52 sec latency 317.876 ms 2024-02-21T23:28:26.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1627 0.49 MB/sec warmup 53 sec latency 327.239 ms 2024-02-21T23:28:26.957 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:28:26.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:28:27.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1646 0.49 MB/sec warmup 54 sec latency 334.242 ms 2024-02-21T23:28:28.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1689 0.48 MB/sec warmup 55 sec latency 307.954 ms 2024-02-21T23:28:29.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1707 0.47 MB/sec warmup 56 sec latency 298.848 ms 2024-02-21T23:28:30.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1772 0.46 MB/sec warmup 57 sec latency 299.436 ms 2024-02-21T23:28:31.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1806 0.45 MB/sec warmup 58 sec latency 318.611 ms 2024-02-21T23:28:32.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1826 0.45 MB/sec warmup 59 sec latency 338.945 ms 2024-02-21T23:28:33.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1841 0.44 MB/sec warmup 60 sec latency 307.863 ms 2024-02-21T23:28:34.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1864 0.43 MB/sec warmup 61 sec latency 316.944 ms 2024-02-21T23:28:35.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1898 0.43 MB/sec warmup 62 sec latency 416.617 ms 2024-02-21T23:28:36.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1912 0.42 MB/sec warmup 63 sec latency 308.377 ms 2024-02-21T23:28:37.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1961 0.41 MB/sec warmup 64 sec latency 337.239 ms 2024-02-21T23:28:38.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 1995 0.41 MB/sec warmup 65 sec latency 311.160 ms 2024-02-21T23:28:39.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2034 0.40 MB/sec warmup 66 sec latency 302.949 ms 2024-02-21T23:28:40.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2052 0.40 MB/sec warmup 67 sec latency 311.542 ms 2024-02-21T23:28:41.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2061 0.39 MB/sec warmup 68 sec latency 305.853 ms 2024-02-21T23:28:42.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2088 0.39 MB/sec warmup 69 sec latency 342.945 ms 2024-02-21T23:28:43.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2108 0.38 MB/sec warmup 70 sec latency 357.355 ms 2024-02-21T23:28:44.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2146 0.38 MB/sec warmup 71 sec latency 308.974 ms 2024-02-21T23:28:45.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2175 0.37 MB/sec warmup 72 sec latency 436.708 ms 2024-02-21T23:28:46.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2198 0.37 MB/sec warmup 73 sec latency 326.873 ms 2024-02-21T23:28:47.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2247 0.36 MB/sec warmup 74 sec latency 317.322 ms 2024-02-21T23:28:48.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2299 0.37 MB/sec warmup 75 sec latency 302.636 ms 2024-02-21T23:28:49.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2307 0.37 MB/sec warmup 76 sec latency 437.246 ms 2024-02-21T23:28:50.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2327 0.36 MB/sec warmup 77 sec latency 360.867 ms 2024-02-21T23:28:51.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2340 0.36 MB/sec warmup 78 sec latency 309.627 ms 2024-02-21T23:28:52.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2362 0.35 MB/sec warmup 79 sec latency 319.375 ms 2024-02-21T23:28:53.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2367 0.35 MB/sec warmup 80 sec latency 547.294 ms 2024-02-21T23:28:54.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2388 0.35 MB/sec warmup 81 sec latency 306.705 ms 2024-02-21T23:28:55.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2401 0.34 MB/sec warmup 82 sec latency 310.215 ms 2024-02-21T23:28:56.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2428 0.34 MB/sec warmup 83 sec latency 312.418 ms 2024-02-21T23:28:57.019 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:28:57.022 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:28:57.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2439 0.33 MB/sec warmup 84 sec latency 308.150 ms 2024-02-21T23:28:58.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2467 0.34 MB/sec warmup 85 sec latency 146.955 ms 2024-02-21T23:28:59.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2492 0.33 MB/sec warmup 86 sec latency 330.342 ms 2024-02-21T23:29:00.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2506 0.33 MB/sec warmup 87 sec latency 320.766 ms 2024-02-21T23:29:01.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2553 0.33 MB/sec warmup 88 sec latency 332.917 ms 2024-02-21T23:29:02.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2574 0.33 MB/sec warmup 89 sec latency 303.279 ms 2024-02-21T23:29:03.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2582 0.32 MB/sec warmup 90 sec latency 349.585 ms 2024-02-21T23:29:04.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2602 0.32 MB/sec warmup 91 sec latency 336.979 ms 2024-02-21T23:29:05.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2617 0.32 MB/sec warmup 92 sec latency 323.554 ms 2024-02-21T23:29:06.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2633 0.32 MB/sec warmup 93 sec latency 321.005 ms 2024-02-21T23:29:07.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2751 0.34 MB/sec warmup 94 sec latency 336.648 ms 2024-02-21T23:29:08.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2770 0.34 MB/sec warmup 95 sec latency 340.529 ms 2024-02-21T23:29:09.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2872 0.34 MB/sec warmup 96 sec latency 305.272 ms 2024-02-21T23:29:10.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2898 0.34 MB/sec warmup 97 sec latency 301.417 ms 2024-02-21T23:29:11.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2931 0.34 MB/sec warmup 98 sec latency 183.177 ms 2024-02-21T23:29:12.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2968 0.34 MB/sec warmup 99 sec latency 315.473 ms 2024-02-21T23:29:13.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2982 0.34 MB/sec warmup 100 sec latency 321.466 ms 2024-02-21T23:29:14.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3007 0.33 MB/sec warmup 101 sec latency 300.456 ms 2024-02-21T23:29:15.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3025 0.33 MB/sec warmup 102 sec latency 211.608 ms 2024-02-21T23:29:16.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3171 0.33 MB/sec warmup 103 sec latency 322.473 ms 2024-02-21T23:29:17.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3204 0.33 MB/sec warmup 104 sec latency 313.358 ms 2024-02-21T23:29:18.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3266 0.34 MB/sec warmup 105 sec latency 309.991 ms 2024-02-21T23:29:19.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3281 0.33 MB/sec warmup 106 sec latency 337.573 ms 2024-02-21T23:29:20.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3285 0.33 MB/sec warmup 107 sec latency 317.604 ms 2024-02-21T23:29:21.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3290 0.33 MB/sec warmup 108 sec latency 352.944 ms 2024-02-21T23:29:22.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3293 0.32 MB/sec warmup 109 sec latency 384.076 ms 2024-02-21T23:29:23.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3296 0.32 MB/sec warmup 110 sec latency 415.848 ms 2024-02-21T23:29:24.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3303 0.32 MB/sec warmup 111 sec latency 306.719 ms 2024-02-21T23:29:25.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3308 0.32 MB/sec warmup 112 sec latency 376.807 ms 2024-02-21T23:29:26.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3312 0.31 MB/sec warmup 113 sec latency 307.131 ms 2024-02-21T23:29:27.128 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:29:27.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:29:27.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3316 0.31 MB/sec warmup 114 sec latency 305.312 ms 2024-02-21T23:29:28.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3319 0.31 MB/sec warmup 115 sec latency 323.966 ms 2024-02-21T23:29:29.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3323 0.30 MB/sec warmup 116 sec latency 318.388 ms 2024-02-21T23:29:30.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3327 0.30 MB/sec warmup 117 sec latency 324.401 ms 2024-02-21T23:29:31.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3332 0.30 MB/sec warmup 118 sec latency 310.501 ms 2024-02-21T23:29:32.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3335 0.30 MB/sec warmup 119 sec latency 307.758 ms 2024-02-21T23:29:34.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3371 0.07 MB/sec execute 1 sec latency 302.696 ms 2024-02-21T23:29:35.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3392 0.05 MB/sec execute 2 sec latency 333.985 ms 2024-02-21T23:29:36.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3410 0.15 MB/sec execute 3 sec latency 331.546 ms 2024-02-21T23:29:37.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3529 0.75 MB/sec execute 4 sec latency 139.015 ms 2024-02-21T23:29:38.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3551 0.61 MB/sec execute 5 sec latency 305.774 ms 2024-02-21T23:29:39.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3569 0.51 MB/sec execute 6 sec latency 322.221 ms 2024-02-21T23:29:40.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3588 0.44 MB/sec execute 7 sec latency 314.023 ms 2024-02-21T23:29:41.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3638 0.41 MB/sec execute 8 sec latency 376.856 ms 2024-02-21T23:29:42.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3666 0.37 MB/sec execute 9 sec latency 331.261 ms 2024-02-21T23:29:43.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3683 0.34 MB/sec execute 10 sec latency 331.611 ms 2024-02-21T23:29:44.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3702 0.31 MB/sec execute 11 sec latency 312.589 ms 2024-02-21T23:29:45.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3758 0.37 MB/sec execute 12 sec latency 123.548 ms 2024-02-21T23:29:46.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3773 0.34 MB/sec execute 13 sec latency 317.158 ms 2024-02-21T23:29:47.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3789 0.31 MB/sec execute 14 sec latency 222.419 ms 2024-02-21T23:29:48.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3797 0.29 MB/sec execute 15 sec latency 190.897 ms 2024-02-21T23:29:49.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3814 0.27 MB/sec execute 16 sec latency 250.495 ms 2024-02-21T23:29:50.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3830 0.26 MB/sec execute 17 sec latency 163.210 ms 2024-02-21T23:29:51.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3846 0.24 MB/sec execute 18 sec latency 151.010 ms 2024-02-21T23:29:52.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3863 0.23 MB/sec execute 19 sec latency 177.282 ms 2024-02-21T23:29:53.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3876 0.22 MB/sec execute 20 sec latency 299.937 ms 2024-02-21T23:29:54.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3896 0.22 MB/sec execute 21 sec latency 326.953 ms 2024-02-21T23:29:55.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3914 0.21 MB/sec execute 22 sec latency 329.487 ms 2024-02-21T23:29:56.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3927 0.20 MB/sec execute 23 sec latency 297.500 ms 2024-02-21T23:29:57.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:29:57.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:29:57.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3939 0.20 MB/sec execute 24 sec latency 174.083 ms 2024-02-21T23:29:58.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3952 0.19 MB/sec execute 25 sec latency 201.553 ms 2024-02-21T23:29:59.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3970 0.18 MB/sec execute 26 sec latency 155.263 ms 2024-02-21T23:30:00.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 3984 0.17 MB/sec execute 27 sec latency 163.922 ms 2024-02-21T23:30:01.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4002 0.17 MB/sec execute 28 sec latency 143.390 ms 2024-02-21T23:30:02.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4020 0.16 MB/sec execute 29 sec latency 147.869 ms 2024-02-21T23:30:03.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4032 0.16 MB/sec execute 30 sec latency 304.007 ms 2024-02-21T23:30:04.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4040 0.15 MB/sec execute 31 sec latency 337.722 ms 2024-02-21T23:30:05.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4059 0.15 MB/sec execute 32 sec latency 323.487 ms 2024-02-21T23:30:06.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4073 0.14 MB/sec execute 33 sec latency 319.913 ms 2024-02-21T23:30:07.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4099 0.14 MB/sec execute 34 sec latency 298.580 ms 2024-02-21T23:30:08.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4120 0.13 MB/sec execute 35 sec latency 223.063 ms 2024-02-21T23:30:09.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4153 0.13 MB/sec execute 36 sec latency 84.660 ms 2024-02-21T23:30:10.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4164 0.13 MB/sec execute 37 sec latency 390.078 ms 2024-02-21T23:30:11.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4176 0.12 MB/sec execute 38 sec latency 307.700 ms 2024-02-21T23:30:12.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4216 0.12 MB/sec execute 39 sec latency 316.619 ms 2024-02-21T23:30:13.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4225 0.12 MB/sec execute 40 sec latency 433.806 ms 2024-02-21T23:30:14.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4252 0.12 MB/sec execute 41 sec latency 343.830 ms 2024-02-21T23:30:15.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4281 0.12 MB/sec execute 42 sec latency 324.739 ms 2024-02-21T23:30:16.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4315 0.12 MB/sec execute 43 sec latency 228.389 ms 2024-02-21T23:30:17.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4349 0.12 MB/sec execute 44 sec latency 331.564 ms 2024-02-21T23:30:18.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4383 0.12 MB/sec execute 45 sec latency 251.747 ms 2024-02-21T23:30:19.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4393 0.11 MB/sec execute 46 sec latency 328.024 ms 2024-02-21T23:30:20.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4424 0.11 MB/sec execute 47 sec latency 461.843 ms 2024-02-21T23:30:21.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4443 0.11 MB/sec execute 48 sec latency 365.665 ms 2024-02-21T23:30:22.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4473 0.11 MB/sec execute 49 sec latency 297.679 ms 2024-02-21T23:30:23.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4498 0.11 MB/sec execute 50 sec latency 312.967 ms 2024-02-21T23:30:24.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4505 0.11 MB/sec execute 51 sec latency 312.597 ms 2024-02-21T23:30:25.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4538 0.11 MB/sec execute 52 sec latency 300.253 ms 2024-02-21T23:30:26.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4563 0.11 MB/sec execute 53 sec latency 280.179 ms 2024-02-21T23:30:27.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:30:27.290 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:30:27.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4602 0.12 MB/sec execute 54 sec latency 314.812 ms 2024-02-21T23:30:28.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4616 0.12 MB/sec execute 55 sec latency 310.194 ms 2024-02-21T23:30:29.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4632 0.12 MB/sec execute 56 sec latency 309.843 ms 2024-02-21T23:30:30.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4667 0.12 MB/sec execute 57 sec latency 479.410 ms 2024-02-21T23:30:31.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4685 0.12 MB/sec execute 58 sec latency 325.220 ms 2024-02-21T23:30:32.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4724 0.11 MB/sec execute 59 sec latency 188.233 ms 2024-02-21T23:30:33.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4735 0.11 MB/sec execute 60 sec latency 315.395 ms 2024-02-21T23:30:34.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4747 0.11 MB/sec execute 61 sec latency 326.724 ms 2024-02-21T23:30:35.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4780 0.11 MB/sec execute 62 sec latency 356.065 ms 2024-02-21T23:30:36.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4814 0.11 MB/sec execute 63 sec latency 193.865 ms 2024-02-21T23:30:37.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4844 0.11 MB/sec execute 64 sec latency 316.601 ms 2024-02-21T23:30:38.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4851 0.11 MB/sec execute 65 sec latency 322.377 ms 2024-02-21T23:30:39.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4946 0.15 MB/sec execute 66 sec latency 303.138 ms 2024-02-21T23:30:40.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4959 0.15 MB/sec execute 67 sec latency 422.132 ms 2024-02-21T23:30:41.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4983 0.14 MB/sec execute 68 sec latency 321.840 ms 2024-02-21T23:30:42.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5012 0.14 MB/sec execute 69 sec latency 308.556 ms 2024-02-21T23:30:43.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5032 0.14 MB/sec execute 70 sec latency 390.028 ms 2024-02-21T23:30:44.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5054 0.14 MB/sec execute 71 sec latency 338.272 ms 2024-02-21T23:30:45.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5078 0.14 MB/sec execute 72 sec latency 310.940 ms 2024-02-21T23:30:46.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5096 0.13 MB/sec execute 73 sec latency 302.210 ms 2024-02-21T23:30:47.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5129 0.13 MB/sec execute 74 sec latency 312.053 ms 2024-02-21T23:30:48.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5154 0.13 MB/sec execute 75 sec latency 163.074 ms 2024-02-21T23:30:49.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5175 0.13 MB/sec execute 76 sec latency 301.759 ms 2024-02-21T23:30:50.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5199 0.13 MB/sec execute 77 sec latency 429.612 ms 2024-02-21T23:30:51.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5218 0.13 MB/sec execute 78 sec latency 321.397 ms 2024-02-21T23:30:52.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5261 0.13 MB/sec execute 79 sec latency 258.877 ms 2024-02-21T23:30:53.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5272 0.12 MB/sec execute 80 sec latency 301.327 ms 2024-02-21T23:30:54.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5337 0.12 MB/sec execute 81 sec latency 306.241 ms 2024-02-21T23:30:55.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5371 0.12 MB/sec execute 82 sec latency 311.320 ms 2024-02-21T23:30:56.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5391 0.12 MB/sec execute 83 sec latency 320.745 ms 2024-02-21T23:30:57.477 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:30:57.479 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:30:57.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5408 0.12 MB/sec execute 84 sec latency 323.274 ms 2024-02-21T23:30:58.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5430 0.12 MB/sec execute 85 sec latency 319.245 ms 2024-02-21T23:30:59.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5463 0.12 MB/sec execute 86 sec latency 322.491 ms 2024-02-21T23:31:00.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5477 0.12 MB/sec execute 87 sec latency 496.550 ms 2024-02-21T23:31:01.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5526 0.12 MB/sec execute 88 sec latency 310.198 ms 2024-02-21T23:31:02.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5554 0.11 MB/sec execute 89 sec latency 311.041 ms 2024-02-21T23:31:03.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5597 0.11 MB/sec execute 90 sec latency 316.333 ms 2024-02-21T23:31:04.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5614 0.11 MB/sec execute 91 sec latency 304.366 ms 2024-02-21T23:31:05.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5623 0.11 MB/sec execute 92 sec latency 308.873 ms 2024-02-21T23:31:06.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5650 0.11 MB/sec execute 93 sec latency 337.752 ms 2024-02-21T23:31:07.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5676 0.11 MB/sec execute 94 sec latency 333.889 ms 2024-02-21T23:31:08.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5708 0.11 MB/sec execute 95 sec latency 298.811 ms 2024-02-21T23:31:09.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5737 0.11 MB/sec execute 96 sec latency 343.307 ms 2024-02-21T23:31:10.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5759 0.11 MB/sec execute 97 sec latency 419.732 ms 2024-02-21T23:31:11.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5812 0.11 MB/sec execute 98 sec latency 311.801 ms 2024-02-21T23:31:12.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5860 0.12 MB/sec execute 99 sec latency 301.548 ms 2024-02-21T23:31:13.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5867 0.12 MB/sec execute 100 sec latency 437.062 ms 2024-02-21T23:31:14.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5892 0.11 MB/sec execute 101 sec latency 327.191 ms 2024-02-21T23:31:15.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5900 0.11 MB/sec execute 102 sec latency 311.050 ms 2024-02-21T23:31:16.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5922 0.11 MB/sec execute 103 sec latency 308.806 ms 2024-02-21T23:31:17.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5927 0.11 MB/sec execute 104 sec latency 515.037 ms 2024-02-21T23:31:18.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5948 0.11 MB/sec execute 105 sec latency 303.786 ms 2024-02-21T23:31:19.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5966 0.11 MB/sec execute 106 sec latency 301.278 ms 2024-02-21T23:31:20.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5988 0.11 MB/sec execute 107 sec latency 356.041 ms 2024-02-21T23:31:21.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 5996 0.11 MB/sec execute 108 sec latency 314.138 ms 2024-02-21T23:31:22.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6026 0.11 MB/sec execute 109 sec latency 143.457 ms 2024-02-21T23:31:23.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6052 0.11 MB/sec execute 110 sec latency 332.804 ms 2024-02-21T23:31:24.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6064 0.11 MB/sec execute 111 sec latency 307.777 ms 2024-02-21T23:31:25.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6112 0.11 MB/sec execute 112 sec latency 342.784 ms 2024-02-21T23:31:26.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6134 0.11 MB/sec execute 113 sec latency 306.925 ms 2024-02-21T23:31:27.578 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:31:27.580 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:31:27.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6141 0.11 MB/sec execute 114 sec latency 341.390 ms 2024-02-21T23:31:28.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6161 0.11 MB/sec execute 115 sec latency 304.881 ms 2024-02-21T23:31:29.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6175 0.11 MB/sec execute 116 sec latency 328.588 ms 2024-02-21T23:31:30.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6191 0.11 MB/sec execute 117 sec latency 463.451 ms 2024-02-21T23:31:31.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6307 0.13 MB/sec execute 118 sec latency 304.402 ms 2024-02-21T23:31:32.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6326 0.13 MB/sec execute 119 sec latency 339.250 ms 2024-02-21T23:31:33.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6427 0.13 MB/sec execute 120 sec latency 308.530 ms 2024-02-21T23:31:34.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6456 0.14 MB/sec execute 121 sec latency 306.041 ms 2024-02-21T23:31:35.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6489 0.14 MB/sec execute 122 sec latency 195.814 ms 2024-02-21T23:31:36.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6524 0.14 MB/sec execute 123 sec latency 311.415 ms 2024-02-21T23:31:37.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6539 0.14 MB/sec execute 124 sec latency 327.218 ms 2024-02-21T23:31:38.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6564 0.14 MB/sec execute 125 sec latency 300.444 ms 2024-02-21T23:31:39.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6586 0.14 MB/sec execute 126 sec latency 218.772 ms 2024-02-21T23:31:40.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6730 0.14 MB/sec execute 127 sec latency 472.924 ms 2024-02-21T23:31:41.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6759 0.14 MB/sec execute 128 sec latency 319.049 ms 2024-02-21T23:31:42.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6820 0.15 MB/sec execute 129 sec latency 302.161 ms 2024-02-21T23:31:43.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6835 0.15 MB/sec execute 130 sec latency 320.020 ms 2024-02-21T23:31:44.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6839 0.15 MB/sec execute 131 sec latency 322.251 ms 2024-02-21T23:31:45.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6844 0.14 MB/sec execute 132 sec latency 369.007 ms 2024-02-21T23:31:46.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6847 0.14 MB/sec execute 133 sec latency 356.271 ms 2024-02-21T23:31:47.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6851 0.14 MB/sec execute 134 sec latency 371.985 ms 2024-02-21T23:31:48.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6858 0.14 MB/sec execute 135 sec latency 378.683 ms 2024-02-21T23:31:49.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6863 0.14 MB/sec execute 136 sec latency 360.073 ms 2024-02-21T23:31:50.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6866 0.14 MB/sec execute 137 sec latency 396.689 ms 2024-02-21T23:31:51.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6869 0.14 MB/sec execute 138 sec latency 315.568 ms 2024-02-21T23:31:52.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6873 0.14 MB/sec execute 139 sec latency 307.648 ms 2024-02-21T23:31:53.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6878 0.14 MB/sec execute 140 sec latency 305.807 ms 2024-02-21T23:31:54.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6881 0.14 MB/sec execute 141 sec latency 316.644 ms 2024-02-21T23:31:55.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6886 0.13 MB/sec execute 142 sec latency 317.546 ms 2024-02-21T23:31:56.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6889 0.13 MB/sec execute 143 sec latency 298.503 ms 2024-02-21T23:31:57.709 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:31:57.712 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:31:57.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6894 0.13 MB/sec execute 144 sec latency 324.018 ms 2024-02-21T23:31:58.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6920 0.13 MB/sec execute 145 sec latency 130.474 ms 2024-02-21T23:31:59.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6946 0.13 MB/sec execute 146 sec latency 199.672 ms 2024-02-21T23:32:00.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6952 0.13 MB/sec execute 147 sec latency 351.196 ms 2024-02-21T23:32:01.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7076 0.15 MB/sec execute 148 sec latency 315.060 ms 2024-02-21T23:32:02.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7101 0.15 MB/sec execute 149 sec latency 305.460 ms 2024-02-21T23:32:03.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7114 0.15 MB/sec execute 150 sec latency 316.770 ms 2024-02-21T23:32:04.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7135 0.15 MB/sec execute 151 sec latency 327.269 ms 2024-02-21T23:32:05.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7185 0.15 MB/sec execute 152 sec latency 324.516 ms 2024-02-21T23:32:06.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7211 0.15 MB/sec execute 153 sec latency 319.332 ms 2024-02-21T23:32:07.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7223 0.15 MB/sec execute 154 sec latency 338.764 ms 2024-02-21T23:32:08.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7242 0.15 MB/sec execute 155 sec latency 310.277 ms 2024-02-21T23:32:09.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7292 0.15 MB/sec execute 156 sec latency 149.352 ms 2024-02-21T23:32:10.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7317 0.15 MB/sec execute 157 sec latency 304.452 ms 2024-02-21T23:32:11.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7333 0.15 MB/sec execute 158 sec latency 186.738 ms 2024-02-21T23:32:12.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7341 0.15 MB/sec execute 159 sec latency 184.012 ms 2024-02-21T23:32:13.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7356 0.15 MB/sec execute 160 sec latency 204.480 ms 2024-02-21T23:32:14.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7372 0.15 MB/sec execute 161 sec latency 163.947 ms 2024-02-21T23:32:15.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7388 0.15 MB/sec execute 162 sec latency 175.408 ms 2024-02-21T23:32:16.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7404 0.14 MB/sec execute 163 sec latency 171.704 ms 2024-02-21T23:32:17.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7418 0.14 MB/sec execute 164 sec latency 310.864 ms 2024-02-21T23:32:18.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7439 0.14 MB/sec execute 165 sec latency 334.601 ms 2024-02-21T23:32:19.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7446 0.14 MB/sec execute 166 sec latency 325.157 ms 2024-02-21T23:32:20.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7470 0.14 MB/sec execute 167 sec latency 303.301 ms 2024-02-21T23:32:21.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7482 0.14 MB/sec execute 168 sec latency 185.342 ms 2024-02-21T23:32:22.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7495 0.14 MB/sec execute 169 sec latency 188.746 ms 2024-02-21T23:32:23.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7511 0.14 MB/sec execute 170 sec latency 148.498 ms 2024-02-21T23:32:24.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7527 0.14 MB/sec execute 171 sec latency 146.057 ms 2024-02-21T23:32:25.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7543 0.14 MB/sec execute 172 sec latency 165.625 ms 2024-02-21T23:32:26.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7559 0.14 MB/sec execute 173 sec latency 157.083 ms 2024-02-21T23:32:27.758 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:32:27.760 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:32:27.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7572 0.14 MB/sec execute 174 sec latency 316.649 ms 2024-02-21T23:32:28.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7580 0.14 MB/sec execute 175 sec latency 325.622 ms 2024-02-21T23:32:29.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7602 0.14 MB/sec execute 176 sec latency 309.798 ms 2024-02-21T23:32:30.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7615 0.13 MB/sec execute 177 sec latency 298.299 ms 2024-02-21T23:32:31.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7637 0.13 MB/sec execute 178 sec latency 312.101 ms 2024-02-21T23:32:32.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7659 0.13 MB/sec execute 179 sec latency 239.368 ms 2024-02-21T23:32:33.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7691 0.13 MB/sec execute 180 sec latency 173.919 ms 2024-02-21T23:32:34.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7704 0.13 MB/sec execute 181 sec latency 410.341 ms 2024-02-21T23:32:35.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7718 0.13 MB/sec execute 182 sec latency 559.313 ms 2024-02-21T23:32:36.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7728 0.13 MB/sec execute 183 sec latency 309.334 ms 2024-02-21T23:32:37.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7763 0.13 MB/sec execute 184 sec latency 367.884 ms 2024-02-21T23:32:38.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7790 0.13 MB/sec execute 185 sec latency 331.640 ms 2024-02-21T23:32:39.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7818 0.13 MB/sec execute 186 sec latency 307.482 ms 2024-02-21T23:32:40.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7857 0.13 MB/sec execute 187 sec latency 216.721 ms 2024-02-21T23:32:41.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7889 0.13 MB/sec execute 188 sec latency 306.705 ms 2024-02-21T23:32:42.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7925 0.13 MB/sec execute 189 sec latency 226.307 ms 2024-02-21T23:32:43.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7930 0.13 MB/sec execute 190 sec latency 347.250 ms 2024-02-21T23:32:44.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7966 0.13 MB/sec execute 191 sec latency 329.042 ms 2024-02-21T23:32:45.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 7975 0.13 MB/sec execute 192 sec latency 550.675 ms 2024-02-21T23:32:46.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8012 0.13 MB/sec execute 193 sec latency 295.958 ms 2024-02-21T23:32:47.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8039 0.13 MB/sec execute 194 sec latency 253.535 ms 2024-02-21T23:32:48.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8044 0.13 MB/sec execute 195 sec latency 323.520 ms 2024-02-21T23:32:49.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8063 0.13 MB/sec execute 196 sec latency 319.334 ms 2024-02-21T23:32:50.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8104 0.13 MB/sec execute 197 sec latency 227.676 ms 2024-02-21T23:32:51.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8139 0.13 MB/sec execute 198 sec latency 340.930 ms 2024-02-21T23:32:52.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8156 0.13 MB/sec execute 199 sec latency 321.407 ms 2024-02-21T23:32:53.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8170 0.13 MB/sec execute 200 sec latency 343.218 ms 2024-02-21T23:32:54.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8201 0.13 MB/sec execute 201 sec latency 295.623 ms 2024-02-21T23:32:55.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8223 0.13 MB/sec execute 202 sec latency 305.071 ms 2024-02-21T23:32:56.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8256 0.13 MB/sec execute 203 sec latency 339.727 ms 2024-02-21T23:32:57.896 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:32:57.898 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:32:57.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8276 0.13 MB/sec execute 204 sec latency 299.157 ms 2024-02-21T23:32:58.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8285 0.13 MB/sec execute 205 sec latency 331.423 ms 2024-02-21T23:32:59.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8312 0.13 MB/sec execute 206 sec latency 309.366 ms 2024-02-21T23:33:00.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8352 0.13 MB/sec execute 207 sec latency 402.165 ms 2024-02-21T23:33:01.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8381 0.12 MB/sec execute 208 sec latency 312.540 ms 2024-02-21T23:33:02.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8390 0.12 MB/sec execute 209 sec latency 347.085 ms 2024-02-21T23:33:03.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8485 0.14 MB/sec execute 210 sec latency 342.361 ms 2024-02-21T23:33:04.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8494 0.14 MB/sec execute 211 sec latency 301.423 ms 2024-02-21T23:33:05.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8506 0.14 MB/sec execute 212 sec latency 387.935 ms 2024-02-21T23:33:06.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8531 0.14 MB/sec execute 213 sec latency 462.595 ms 2024-02-21T23:33:07.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8557 0.13 MB/sec execute 214 sec latency 356.852 ms 2024-02-21T23:33:08.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8592 0.13 MB/sec execute 215 sec latency 362.206 ms 2024-02-21T23:33:09.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8599 0.13 MB/sec execute 216 sec latency 334.083 ms 2024-02-21T23:33:10.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8634 0.13 MB/sec execute 217 sec latency 320.146 ms 2024-02-21T23:33:11.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8661 0.13 MB/sec execute 218 sec latency 296.370 ms 2024-02-21T23:33:12.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8686 0.13 MB/sec execute 219 sec latency 318.966 ms 2024-02-21T23:33:13.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8711 0.13 MB/sec execute 220 sec latency 314.430 ms 2024-02-21T23:33:14.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8724 0.13 MB/sec execute 221 sec latency 311.051 ms 2024-02-21T23:33:15.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8749 0.13 MB/sec execute 222 sec latency 391.261 ms 2024-02-21T23:33:16.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8771 0.13 MB/sec execute 223 sec latency 427.084 ms 2024-02-21T23:33:17.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8809 0.13 MB/sec execute 224 sec latency 295.614 ms 2024-02-21T23:33:18.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8860 0.13 MB/sec execute 225 sec latency 314.850 ms 2024-02-21T23:33:19.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8901 0.13 MB/sec execute 226 sec latency 306.161 ms 2024-02-21T23:33:20.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8929 0.13 MB/sec execute 227 sec latency 334.784 ms 2024-02-21T23:33:21.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8945 0.13 MB/sec execute 228 sec latency 325.093 ms 2024-02-21T23:33:22.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8967 0.13 MB/sec execute 229 sec latency 308.051 ms 2024-02-21T23:33:23.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8978 0.13 MB/sec execute 230 sec latency 303.737 ms 2024-02-21T23:33:24.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9012 0.13 MB/sec execute 231 sec latency 364.234 ms 2024-02-21T23:33:25.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9029 0.13 MB/sec execute 232 sec latency 425.128 ms 2024-02-21T23:33:26.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9084 0.13 MB/sec execute 233 sec latency 312.961 ms 2024-02-21T23:33:27.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9108 0.13 MB/sec execute 234 sec latency 327.673 ms 2024-02-21T23:33:28.035 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:33:28.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:33:28.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9149 0.13 MB/sec execute 235 sec latency 323.530 ms 2024-02-21T23:33:29.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9163 0.12 MB/sec execute 236 sec latency 324.583 ms 2024-02-21T23:33:30.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9185 0.12 MB/sec execute 237 sec latency 337.837 ms 2024-02-21T23:33:31.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9202 0.12 MB/sec execute 238 sec latency 313.791 ms 2024-02-21T23:33:32.011 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:08:59.920 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:33:32.068 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:08:59.976 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:33:32.100 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:09:00.012 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:33:32.110 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:09:00.023 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:33:32.163 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:09:00.075 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:33:32.174 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:09:00.086 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:33:32.430 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:09:00.338 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:33:32.551 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:09:00.464 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:33:32.563 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:09:00.476 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:33:32.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9246 0.12 MB/sec execute 239 sec latency 375.830 ms 2024-02-21T23:33:33.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9279 0.12 MB/sec execute 240 sec latency 365.221 ms 2024-02-21T23:33:34.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9287 0.12 MB/sec execute 241 sec latency 336.679 ms 2024-02-21T23:33:35.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9307 0.12 MB/sec execute 242 sec latency 336.098 ms 2024-02-21T23:33:36.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9369 0.12 MB/sec execute 243 sec latency 313.493 ms 2024-02-21T23:33:37.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9407 0.13 MB/sec execute 244 sec latency 296.846 ms 2024-02-21T23:33:38.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9425 0.13 MB/sec execute 245 sec latency 424.000 ms 2024-02-21T23:33:39.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9439 0.12 MB/sec execute 246 sec latency 306.645 ms 2024-02-21T23:33:40.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9462 0.12 MB/sec execute 247 sec latency 309.375 ms 2024-02-21T23:33:41.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9468 0.12 MB/sec execute 248 sec latency 310.707 ms 2024-02-21T23:33:42.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9475 0.12 MB/sec execute 249 sec latency 563.948 ms 2024-02-21T23:33:43.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9495 0.12 MB/sec execute 250 sec latency 334.662 ms 2024-02-21T23:33:44.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9514 0.12 MB/sec execute 251 sec latency 300.338 ms 2024-02-21T23:33:45.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9535 0.12 MB/sec execute 252 sec latency 325.029 ms 2024-02-21T23:33:46.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9545 0.12 MB/sec execute 253 sec latency 323.687 ms 2024-02-21T23:33:47.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9574 0.12 MB/sec execute 254 sec latency 168.166 ms 2024-02-21T23:33:48.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9598 0.12 MB/sec execute 255 sec latency 336.097 ms 2024-02-21T23:33:49.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9651 0.12 MB/sec execute 256 sec latency 316.609 ms 2024-02-21T23:33:50.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9662 0.12 MB/sec execute 257 sec latency 323.938 ms 2024-02-21T23:33:51.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9682 0.12 MB/sec execute 258 sec latency 309.817 ms 2024-02-21T23:33:52.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9698 0.12 MB/sec execute 259 sec latency 341.919 ms 2024-02-21T23:33:53.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9710 0.12 MB/sec execute 260 sec latency 329.608 ms 2024-02-21T23:33:54.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9720 0.12 MB/sec execute 261 sec latency 321.729 ms 2024-02-21T23:33:55.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9819 0.13 MB/sec execute 262 sec latency 366.023 ms 2024-02-21T23:33:56.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9860 0.13 MB/sec execute 263 sec latency 411.628 ms 2024-02-21T23:33:57.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9880 0.13 MB/sec execute 264 sec latency 328.598 ms 2024-02-21T23:33:58.390 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:33:58.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:33:58.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 9978 0.13 MB/sec execute 265 sec latency 308.969 ms 2024-02-21T23:33:59.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10017 0.14 MB/sec execute 266 sec latency 148.990 ms 2024-02-21T23:34:00.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10037 0.14 MB/sec execute 267 sec latency 311.382 ms 2024-02-21T23:34:01.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10073 0.14 MB/sec execute 268 sec latency 320.085 ms 2024-02-21T23:34:02.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10093 0.14 MB/sec execute 269 sec latency 353.028 ms 2024-02-21T23:34:03.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10116 0.13 MB/sec execute 270 sec latency 315.947 ms 2024-02-21T23:34:04.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10266 0.14 MB/sec execute 271 sec latency 315.582 ms 2024-02-21T23:34:05.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10287 0.14 MB/sec execute 272 sec latency 315.432 ms 2024-02-21T23:34:06.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10323 0.14 MB/sec execute 273 sec latency 321.885 ms 2024-02-21T23:34:07.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10372 0.14 MB/sec execute 274 sec latency 308.828 ms 2024-02-21T23:34:08.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10381 0.14 MB/sec execute 275 sec latency 319.687 ms 2024-02-21T23:34:09.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10385 0.14 MB/sec execute 276 sec latency 325.696 ms 2024-02-21T23:34:10.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10390 0.14 MB/sec execute 277 sec latency 368.524 ms 2024-02-21T23:34:11.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10393 0.14 MB/sec execute 278 sec latency 358.993 ms 2024-02-21T23:34:12.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10398 0.14 MB/sec execute 279 sec latency 412.989 ms 2024-02-21T23:34:13.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10406 0.14 MB/sec execute 280 sec latency 342.246 ms 2024-02-21T23:34:14.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10409 0.14 MB/sec execute 281 sec latency 301.645 ms 2024-02-21T23:34:15.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10412 0.14 MB/sec execute 282 sec latency 381.321 ms 2024-02-21T23:34:16.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10416 0.14 MB/sec execute 283 sec latency 312.790 ms 2024-02-21T23:34:17.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10421 0.13 MB/sec execute 284 sec latency 313.552 ms 2024-02-21T23:34:18.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10424 0.13 MB/sec execute 285 sec latency 305.357 ms 2024-02-21T23:34:19.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10429 0.13 MB/sec execute 286 sec latency 303.542 ms 2024-02-21T23:34:20.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10432 0.13 MB/sec execute 287 sec latency 315.051 ms 2024-02-21T23:34:21.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10435 0.13 MB/sec execute 288 sec latency 314.003 ms 2024-02-21T23:34:22.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10458 0.13 MB/sec execute 289 sec latency 305.599 ms 2024-02-21T23:34:23.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10477 0.13 MB/sec execute 290 sec latency 129.463 ms 2024-02-21T23:34:24.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10493 0.13 MB/sec execute 291 sec latency 311.393 ms 2024-02-21T23:34:25.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10601 0.14 MB/sec execute 292 sec latency 317.033 ms 2024-02-21T23:34:26.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10629 0.14 MB/sec execute 293 sec latency 300.207 ms 2024-02-21T23:34:27.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10653 0.14 MB/sec execute 294 sec latency 303.812 ms 2024-02-21T23:34:28.511 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:34:28.514 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:34:28.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10669 0.14 MB/sec execute 295 sec latency 323.204 ms 2024-02-21T23:34:29.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10718 0.14 MB/sec execute 296 sec latency 386.995 ms 2024-02-21T23:34:30.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10733 0.14 MB/sec execute 297 sec latency 230.914 ms 2024-02-21T23:34:31.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10763 0.14 MB/sec execute 298 sec latency 333.183 ms 2024-02-21T23:34:32.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10776 0.14 MB/sec execute 299 sec latency 318.176 ms 2024-02-21T23:34:33.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10805 0.14 MB/sec execute 300 sec latency 319.707 ms 2024-02-21T23:34:34.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10849 0.14 MB/sec execute 301 sec latency 302.192 ms 2024-02-21T23:34:35.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10869 0.14 MB/sec execute 302 sec latency 236.238 ms 2024-02-21T23:34:36.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10881 0.14 MB/sec execute 303 sec latency 172.714 ms 2024-02-21T23:34:37.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10891 0.14 MB/sec execute 304 sec latency 208.052 ms 2024-02-21T23:34:38.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10909 0.14 MB/sec execute 305 sec latency 170.345 ms 2024-02-21T23:34:39.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10925 0.14 MB/sec execute 306 sec latency 196.611 ms 2024-02-21T23:34:40.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10941 0.14 MB/sec execute 307 sec latency 261.719 ms 2024-02-21T23:34:41.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10958 0.14 MB/sec execute 308 sec latency 137.258 ms 2024-02-21T23:34:42.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10967 0.14 MB/sec execute 309 sec latency 329.239 ms 2024-02-21T23:34:43.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 10986 0.14 MB/sec execute 310 sec latency 325.710 ms 2024-02-21T23:34:44.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11004 0.14 MB/sec execute 311 sec latency 338.115 ms 2024-02-21T23:34:45.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11018 0.14 MB/sec execute 312 sec latency 329.706 ms 2024-02-21T23:34:46.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11029 0.14 MB/sec execute 313 sec latency 171.677 ms 2024-02-21T23:34:47.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11044 0.14 MB/sec execute 314 sec latency 212.560 ms 2024-02-21T23:34:48.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11061 0.14 MB/sec execute 315 sec latency 159.061 ms 2024-02-21T23:34:49.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11077 0.14 MB/sec execute 316 sec latency 150.732 ms 2024-02-21T23:34:50.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11095 0.14 MB/sec execute 317 sec latency 159.760 ms 2024-02-21T23:34:51.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11112 0.13 MB/sec execute 318 sec latency 142.278 ms 2024-02-21T23:34:52.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11122 0.13 MB/sec execute 319 sec latency 337.666 ms 2024-02-21T23:34:53.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11132 0.13 MB/sec execute 320 sec latency 339.110 ms 2024-02-21T23:34:54.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11150 0.13 MB/sec execute 321 sec latency 324.033 ms 2024-02-21T23:34:55.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11171 0.13 MB/sec execute 322 sec latency 306.557 ms 2024-02-21T23:34:56.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11192 0.13 MB/sec execute 323 sec latency 303.611 ms 2024-02-21T23:34:57.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11213 0.13 MB/sec execute 324 sec latency 236.557 ms 2024-02-21T23:34:58.633 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:34:58.635 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:34:58.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11242 0.13 MB/sec execute 325 sec latency 89.508 ms 2024-02-21T23:34:59.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11257 0.13 MB/sec execute 326 sec latency 379.770 ms 2024-02-21T23:35:00.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11265 0.13 MB/sec execute 327 sec latency 460.030 ms 2024-02-21T23:35:01.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11305 0.13 MB/sec execute 328 sec latency 485.110 ms 2024-02-21T23:35:02.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11314 0.13 MB/sec execute 329 sec latency 347.614 ms 2024-02-21T23:35:03.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11341 0.13 MB/sec execute 330 sec latency 343.741 ms 2024-02-21T23:35:04.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11370 0.13 MB/sec execute 331 sec latency 331.090 ms 2024-02-21T23:35:05.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11404 0.13 MB/sec execute 332 sec latency 243.987 ms 2024-02-21T23:35:06.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11438 0.13 MB/sec execute 333 sec latency 339.091 ms 2024-02-21T23:35:07.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11472 0.13 MB/sec execute 334 sec latency 226.882 ms 2024-02-21T23:35:08.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11482 0.13 MB/sec execute 335 sec latency 310.456 ms 2024-02-21T23:35:09.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11513 0.13 MB/sec execute 336 sec latency 337.815 ms 2024-02-21T23:35:10.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11532 0.13 MB/sec execute 337 sec latency 329.111 ms 2024-02-21T23:35:11.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11561 0.13 MB/sec execute 338 sec latency 314.440 ms 2024-02-21T23:35:12.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11587 0.13 MB/sec execute 339 sec latency 306.407 ms 2024-02-21T23:35:13.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11593 0.13 MB/sec execute 340 sec latency 324.110 ms 2024-02-21T23:35:14.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11625 0.13 MB/sec execute 341 sec latency 314.694 ms 2024-02-21T23:35:15.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11652 0.13 MB/sec execute 342 sec latency 245.893 ms 2024-02-21T23:35:16.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11691 0.13 MB/sec execute 343 sec latency 363.043 ms 2024-02-21T23:35:17.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11704 0.13 MB/sec execute 344 sec latency 319.773 ms 2024-02-21T23:35:18.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11721 0.13 MB/sec execute 345 sec latency 311.765 ms 2024-02-21T23:35:19.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11757 0.13 MB/sec execute 346 sec latency 313.394 ms 2024-02-21T23:35:20.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11779 0.13 MB/sec execute 347 sec latency 316.953 ms 2024-02-21T23:35:21.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11814 0.13 MB/sec execute 348 sec latency 191.694 ms 2024-02-21T23:35:22.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11826 0.13 MB/sec execute 349 sec latency 313.141 ms 2024-02-21T23:35:23.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11837 0.13 MB/sec execute 350 sec latency 316.708 ms 2024-02-21T23:35:24.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11871 0.13 MB/sec execute 351 sec latency 314.653 ms 2024-02-21T23:35:25.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11904 0.13 MB/sec execute 352 sec latency 248.549 ms 2024-02-21T23:35:26.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11934 0.13 MB/sec execute 353 sec latency 371.209 ms 2024-02-21T23:35:27.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11940 0.13 MB/sec execute 354 sec latency 353.627 ms 2024-02-21T23:35:28.696 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:35:28.699 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:35:28.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12034 0.14 MB/sec execute 355 sec latency 338.087 ms 2024-02-21T23:35:29.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12049 0.13 MB/sec execute 356 sec latency 312.215 ms 2024-02-21T23:35:30.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12073 0.13 MB/sec execute 357 sec latency 316.703 ms 2024-02-21T23:35:31.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12102 0.13 MB/sec execute 358 sec latency 343.121 ms 2024-02-21T23:35:32.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12122 0.13 MB/sec execute 359 sec latency 300.276 ms 2024-02-21T23:35:33.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12145 0.13 MB/sec execute 360 sec latency 338.813 ms 2024-02-21T23:35:34.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12168 0.13 MB/sec execute 361 sec latency 330.320 ms 2024-02-21T23:35:35.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12186 0.13 MB/sec execute 362 sec latency 301.651 ms 2024-02-21T23:35:36.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12217 0.13 MB/sec execute 363 sec latency 353.876 ms 2024-02-21T23:35:37.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12244 0.13 MB/sec execute 364 sec latency 166.981 ms 2024-02-21T23:35:38.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12265 0.13 MB/sec execute 365 sec latency 323.553 ms 2024-02-21T23:35:39.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12296 0.13 MB/sec execute 366 sec latency 316.726 ms 2024-02-21T23:35:40.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12308 0.13 MB/sec execute 367 sec latency 342.801 ms 2024-02-21T23:35:41.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12344 0.13 MB/sec execute 368 sec latency 280.996 ms 2024-02-21T23:35:42.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12362 0.13 MB/sec execute 369 sec latency 305.459 ms 2024-02-21T23:35:43.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12427 0.13 MB/sec execute 370 sec latency 316.429 ms 2024-02-21T23:35:44.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12462 0.13 MB/sec execute 371 sec latency 316.215 ms 2024-02-21T23:35:45.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12482 0.13 MB/sec execute 372 sec latency 323.963 ms 2024-02-21T23:35:46.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12499 0.13 MB/sec execute 373 sec latency 307.930 ms 2024-02-21T23:35:47.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12523 0.13 MB/sec execute 374 sec latency 304.535 ms 2024-02-21T23:35:48.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12556 0.13 MB/sec execute 375 sec latency 334.722 ms 2024-02-21T23:35:49.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12570 0.13 MB/sec execute 376 sec latency 317.146 ms 2024-02-21T23:35:50.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12620 0.13 MB/sec execute 377 sec latency 300.602 ms 2024-02-21T23:35:51.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12655 0.13 MB/sec execute 378 sec latency 304.100 ms 2024-02-21T23:35:52.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12690 0.13 MB/sec execute 379 sec latency 308.445 ms 2024-02-21T23:35:53.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12708 0.13 MB/sec execute 380 sec latency 336.368 ms 2024-02-21T23:35:54.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12727 0.13 MB/sec execute 381 sec latency 328.403 ms 2024-02-21T23:35:55.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12748 0.13 MB/sec execute 382 sec latency 327.583 ms 2024-02-21T23:35:56.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12792 0.13 MB/sec execute 383 sec latency 345.949 ms 2024-02-21T23:35:57.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12807 0.13 MB/sec execute 384 sec latency 354.235 ms 2024-02-21T23:35:58.841 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:35:58.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:35:58.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12832 0.13 MB/sec execute 385 sec latency 335.590 ms 2024-02-21T23:35:59.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12854 0.13 MB/sec execute 386 sec latency 319.953 ms 2024-02-21T23:36:00.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12912 0.13 MB/sec execute 387 sec latency 314.531 ms 2024-02-21T23:36:01.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12954 0.13 MB/sec execute 388 sec latency 311.763 ms 2024-02-21T23:36:02.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12961 0.13 MB/sec execute 389 sec latency 322.098 ms 2024-02-21T23:36:03.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12987 0.13 MB/sec execute 390 sec latency 376.934 ms 2024-02-21T23:36:04.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 12994 0.13 MB/sec execute 391 sec latency 300.616 ms 2024-02-21T23:36:05.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13016 0.13 MB/sec execute 392 sec latency 311.927 ms 2024-02-21T23:36:06.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13020 0.13 MB/sec execute 393 sec latency 579.954 ms 2024-02-21T23:36:07.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13042 0.13 MB/sec execute 394 sec latency 338.602 ms 2024-02-21T23:36:08.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13055 0.13 MB/sec execute 395 sec latency 315.661 ms 2024-02-21T23:36:09.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13082 0.13 MB/sec execute 396 sec latency 310.648 ms 2024-02-21T23:36:10.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13090 0.13 MB/sec execute 397 sec latency 324.251 ms 2024-02-21T23:36:11.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13120 0.13 MB/sec execute 398 sec latency 132.693 ms 2024-02-21T23:36:12.050 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:11:39.958 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:36:12.053 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:11:39.962 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:36:12.067 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:11:39.979 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:36:12.069 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:11:39.982 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:36:12.071 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:11:39.984 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:36:12.074 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:11:39.987 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:36:12.137 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:11:40.045 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:36:12.157 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:11:40.070 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:36:12.164 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:11:40.076 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:36:12.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13143 0.13 MB/sec execute 399 sec latency 364.778 ms 2024-02-21T23:36:13.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13154 0.13 MB/sec execute 400 sec latency 326.116 ms 2024-02-21T23:36:14.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13203 0.13 MB/sec execute 401 sec latency 313.150 ms 2024-02-21T23:36:15.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13223 0.13 MB/sec execute 402 sec latency 313.815 ms 2024-02-21T23:36:16.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13234 0.13 MB/sec execute 403 sec latency 324.567 ms 2024-02-21T23:36:17.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13254 0.13 MB/sec execute 404 sec latency 303.273 ms 2024-02-21T23:36:18.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13269 0.13 MB/sec execute 405 sec latency 338.792 ms 2024-02-21T23:36:19.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13307 0.13 MB/sec execute 406 sec latency 353.135 ms 2024-02-21T23:36:20.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13401 0.13 MB/sec execute 407 sec latency 310.908 ms 2024-02-21T23:36:21.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13421 0.13 MB/sec execute 408 sec latency 309.018 ms 2024-02-21T23:36:22.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13522 0.13 MB/sec execute 409 sec latency 326.621 ms 2024-02-21T23:36:23.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13548 0.13 MB/sec execute 410 sec latency 317.469 ms 2024-02-21T23:36:24.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13582 0.13 MB/sec execute 411 sec latency 197.215 ms 2024-02-21T23:36:25.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13618 0.13 MB/sec execute 412 sec latency 308.556 ms 2024-02-21T23:36:26.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13634 0.13 MB/sec execute 413 sec latency 351.566 ms 2024-02-21T23:36:27.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13657 0.13 MB/sec execute 414 sec latency 300.733 ms 2024-02-21T23:36:28.904 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:36:28.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:36:28.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13681 0.13 MB/sec execute 415 sec latency 232.792 ms 2024-02-21T23:36:29.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13826 0.13 MB/sec execute 416 sec latency 337.820 ms 2024-02-21T23:36:30.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13854 0.13 MB/sec execute 417 sec latency 312.156 ms 2024-02-21T23:36:31.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13917 0.14 MB/sec execute 418 sec latency 314.746 ms 2024-02-21T23:36:32.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13930 0.14 MB/sec execute 419 sec latency 307.352 ms 2024-02-21T23:36:33.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13934 0.14 MB/sec execute 420 sec latency 316.184 ms 2024-02-21T23:36:34.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13939 0.14 MB/sec execute 421 sec latency 343.462 ms 2024-02-21T23:36:35.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13942 0.14 MB/sec execute 422 sec latency 331.339 ms 2024-02-21T23:36:36.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13947 0.14 MB/sec execute 423 sec latency 394.032 ms 2024-02-21T23:36:37.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13954 0.14 MB/sec execute 424 sec latency 409.283 ms 2024-02-21T23:36:38.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13957 0.13 MB/sec execute 425 sec latency 351.284 ms 2024-02-21T23:36:39.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13960 0.13 MB/sec execute 426 sec latency 339.114 ms 2024-02-21T23:36:40.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13963 0.13 MB/sec execute 427 sec latency 334.355 ms 2024-02-21T23:36:41.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13967 0.13 MB/sec execute 428 sec latency 445.360 ms 2024-02-21T23:36:42.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13972 0.13 MB/sec execute 429 sec latency 309.571 ms 2024-02-21T23:36:43.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13975 0.13 MB/sec execute 430 sec latency 309.451 ms 2024-02-21T23:36:44.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13980 0.13 MB/sec execute 431 sec latency 309.138 ms 2024-02-21T23:36:45.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13983 0.13 MB/sec execute 432 sec latency 343.515 ms 2024-02-21T23:36:46.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13986 0.13 MB/sec execute 433 sec latency 310.652 ms 2024-02-21T23:36:47.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14014 0.13 MB/sec execute 434 sec latency 324.067 ms 2024-02-21T23:36:48.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14040 0.13 MB/sec execute 435 sec latency 137.191 ms 2024-02-21T23:36:49.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14047 0.13 MB/sec execute 436 sec latency 335.307 ms 2024-02-21T23:36:50.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14164 0.14 MB/sec execute 437 sec latency 313.466 ms 2024-02-21T23:36:51.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14194 0.14 MB/sec execute 438 sec latency 299.520 ms 2024-02-21T23:36:52.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14209 0.14 MB/sec execute 439 sec latency 317.546 ms 2024-02-21T23:36:53.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14230 0.14 MB/sec execute 440 sec latency 348.148 ms 2024-02-21T23:36:54.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14277 0.14 MB/sec execute 441 sec latency 366.397 ms 2024-02-21T23:36:55.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14305 0.14 MB/sec execute 442 sec latency 306.189 ms 2024-02-21T23:36:56.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14318 0.14 MB/sec execute 443 sec latency 307.100 ms 2024-02-21T23:36:57.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14337 0.14 MB/sec execute 444 sec latency 307.059 ms 2024-02-21T23:36:58.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:36:58.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:36:58.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14378 0.14 MB/sec execute 445 sec latency 130.463 ms 2024-02-21T23:36:59.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14412 0.14 MB/sec execute 446 sec latency 301.881 ms 2024-02-21T23:37:00.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14427 0.14 MB/sec execute 447 sec latency 198.469 ms 2024-02-21T23:37:01.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14436 0.14 MB/sec execute 448 sec latency 194.739 ms 2024-02-21T23:37:02.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14451 0.14 MB/sec execute 449 sec latency 172.701 ms 2024-02-21T23:37:03.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14467 0.14 MB/sec execute 450 sec latency 165.735 ms 2024-02-21T23:37:04.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14485 0.14 MB/sec execute 451 sec latency 171.064 ms 2024-02-21T23:37:05.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14500 0.14 MB/sec execute 452 sec latency 246.192 ms 2024-02-21T23:37:06.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14515 0.14 MB/sec execute 453 sec latency 307.304 ms 2024-02-21T23:37:07.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14535 0.14 MB/sec execute 454 sec latency 332.930 ms 2024-02-21T23:37:08.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14545 0.14 MB/sec execute 455 sec latency 323.910 ms 2024-02-21T23:37:09.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14566 0.14 MB/sec execute 456 sec latency 307.025 ms 2024-02-21T23:37:10.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14577 0.14 MB/sec execute 457 sec latency 190.441 ms 2024-02-21T23:37:11.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14590 0.14 MB/sec execute 458 sec latency 220.000 ms 2024-02-21T23:37:12.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14606 0.14 MB/sec execute 459 sec latency 153.852 ms 2024-02-21T23:37:13.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14623 0.13 MB/sec execute 460 sec latency 153.921 ms 2024-02-21T23:37:14.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14640 0.13 MB/sec execute 461 sec latency 149.047 ms 2024-02-21T23:37:15.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14657 0.13 MB/sec execute 462 sec latency 141.598 ms 2024-02-21T23:37:16.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14670 0.13 MB/sec execute 463 sec latency 302.231 ms 2024-02-21T23:37:17.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14679 0.13 MB/sec execute 464 sec latency 326.590 ms 2024-02-21T23:37:18.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14698 0.13 MB/sec execute 465 sec latency 304.977 ms 2024-02-21T23:37:19.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14715 0.13 MB/sec execute 466 sec latency 293.713 ms 2024-02-21T23:37:20.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14737 0.13 MB/sec execute 467 sec latency 290.361 ms 2024-02-21T23:37:21.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14759 0.13 MB/sec execute 468 sec latency 209.855 ms 2024-02-21T23:37:22.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14789 0.13 MB/sec execute 469 sec latency 90.430 ms 2024-02-21T23:37:23.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14804 0.13 MB/sec execute 470 sec latency 387.284 ms 2024-02-21T23:37:24.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14815 0.13 MB/sec execute 471 sec latency 371.549 ms 2024-02-21T23:37:25.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14852 0.13 MB/sec execute 472 sec latency 344.224 ms 2024-02-21T23:37:26.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14860 0.13 MB/sec execute 473 sec latency 528.127 ms 2024-02-21T23:37:27.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14888 0.13 MB/sec execute 474 sec latency 330.997 ms 2024-02-21T23:37:28.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14917 0.13 MB/sec execute 475 sec latency 302.967 ms 2024-02-21T23:37:29.317 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:37:29.320 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:37:29.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14951 0.13 MB/sec execute 476 sec latency 234.909 ms 2024-02-21T23:37:30.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14985 0.13 MB/sec execute 477 sec latency 341.971 ms 2024-02-21T23:37:31.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15019 0.13 MB/sec execute 478 sec latency 215.791 ms 2024-02-21T23:37:32.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15029 0.13 MB/sec execute 479 sec latency 323.761 ms 2024-02-21T23:37:33.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15063 0.13 MB/sec execute 480 sec latency 303.908 ms 2024-02-21T23:37:34.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15086 0.13 MB/sec execute 481 sec latency 363.173 ms 2024-02-21T23:37:35.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15109 0.13 MB/sec execute 482 sec latency 326.480 ms 2024-02-21T23:37:36.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15134 0.13 MB/sec execute 483 sec latency 318.663 ms 2024-02-21T23:37:37.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15140 0.13 MB/sec execute 484 sec latency 329.344 ms 2024-02-21T23:37:38.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15171 0.13 MB/sec execute 485 sec latency 305.131 ms 2024-02-21T23:37:39.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15199 0.13 MB/sec execute 486 sec latency 244.743 ms 2024-02-21T23:37:40.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15238 0.13 MB/sec execute 487 sec latency 333.604 ms 2024-02-21T23:37:41.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15251 0.13 MB/sec execute 488 sec latency 319.953 ms 2024-02-21T23:37:42.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15267 0.13 MB/sec execute 489 sec latency 319.006 ms 2024-02-21T23:37:43.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15303 0.13 MB/sec execute 490 sec latency 311.903 ms 2024-02-21T23:37:44.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15326 0.13 MB/sec execute 491 sec latency 318.787 ms 2024-02-21T23:37:45.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15360 0.13 MB/sec execute 492 sec latency 203.843 ms 2024-02-21T23:37:46.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15371 0.13 MB/sec execute 493 sec latency 475.061 ms 2024-02-21T23:37:47.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15382 0.13 MB/sec execute 494 sec latency 307.889 ms 2024-02-21T23:37:48.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15416 0.13 MB/sec execute 495 sec latency 373.043 ms 2024-02-21T23:37:49.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15451 0.13 MB/sec execute 496 sec latency 221.126 ms 2024-02-21T23:37:50.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15480 0.13 MB/sec execute 497 sec latency 326.952 ms 2024-02-21T23:37:51.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15486 0.13 MB/sec execute 498 sec latency 424.320 ms 2024-02-21T23:37:52.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15579 0.13 MB/sec execute 499 sec latency 320.682 ms 2024-02-21T23:37:53.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15591 0.13 MB/sec execute 500 sec latency 323.716 ms 2024-02-21T23:37:54.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15619 0.13 MB/sec execute 501 sec latency 315.505 ms 2024-02-21T23:37:55.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15648 0.13 MB/sec execute 502 sec latency 313.029 ms 2024-02-21T23:37:56.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15667 0.13 MB/sec execute 503 sec latency 348.320 ms 2024-02-21T23:37:57.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15689 0.13 MB/sec execute 504 sec latency 334.918 ms 2024-02-21T23:37:58.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15711 0.13 MB/sec execute 505 sec latency 319.633 ms 2024-02-21T23:37:59.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:37:59.424 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:37:59.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15732 0.13 MB/sec execute 506 sec latency 289.583 ms 2024-02-21T23:38:00.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15757 0.13 MB/sec execute 507 sec latency 313.823 ms 2024-02-21T23:38:01.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15790 0.13 MB/sec execute 508 sec latency 96.425 ms 2024-02-21T23:38:02.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15811 0.13 MB/sec execute 509 sec latency 314.590 ms 2024-02-21T23:38:03.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15831 0.13 MB/sec execute 510 sec latency 321.148 ms 2024-02-21T23:38:04.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15852 0.13 MB/sec execute 511 sec latency 318.399 ms 2024-02-21T23:38:05.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15887 0.13 MB/sec execute 512 sec latency 382.921 ms 2024-02-21T23:38:06.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15908 0.13 MB/sec execute 513 sec latency 465.200 ms 2024-02-21T23:38:07.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 15971 0.13 MB/sec execute 514 sec latency 331.953 ms 2024-02-21T23:38:08.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16006 0.13 MB/sec execute 515 sec latency 300.767 ms 2024-02-21T23:38:09.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16023 0.13 MB/sec execute 516 sec latency 333.271 ms 2024-02-21T23:38:10.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16042 0.13 MB/sec execute 517 sec latency 325.687 ms 2024-02-21T23:38:11.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16063 0.13 MB/sec execute 518 sec latency 317.850 ms 2024-02-21T23:38:12.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16089 0.13 MB/sec execute 519 sec latency 324.328 ms 2024-02-21T23:38:13.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16111 0.13 MB/sec execute 520 sec latency 359.995 ms 2024-02-21T23:38:14.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16155 0.13 MB/sec execute 521 sec latency 306.770 ms 2024-02-21T23:38:15.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16187 0.13 MB/sec execute 522 sec latency 311.030 ms 2024-02-21T23:38:16.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16232 0.13 MB/sec execute 523 sec latency 296.990 ms 2024-02-21T23:38:17.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16249 0.13 MB/sec execute 524 sec latency 313.875 ms 2024-02-21T23:38:18.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16259 0.13 MB/sec execute 525 sec latency 303.387 ms 2024-02-21T23:38:19.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16286 0.13 MB/sec execute 526 sec latency 344.984 ms 2024-02-21T23:38:20.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16302 0.13 MB/sec execute 527 sec latency 364.741 ms 2024-02-21T23:38:21.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16344 0.13 MB/sec execute 528 sec latency 310.588 ms 2024-02-21T23:38:22.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16373 0.13 MB/sec execute 529 sec latency 355.895 ms 2024-02-21T23:38:23.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16395 0.13 MB/sec execute 530 sec latency 371.219 ms 2024-02-21T23:38:24.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16444 0.13 MB/sec execute 531 sec latency 316.288 ms 2024-02-21T23:38:25.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16492 0.13 MB/sec execute 532 sec latency 308.766 ms 2024-02-21T23:38:26.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16502 0.13 MB/sec execute 533 sec latency 390.255 ms 2024-02-21T23:38:27.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16520 0.13 MB/sec execute 534 sec latency 454.541 ms 2024-02-21T23:38:28.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16536 0.13 MB/sec execute 535 sec latency 330.153 ms 2024-02-21T23:38:29.568 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:38:29.570 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:38:29.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16557 0.13 MB/sec execute 536 sec latency 330.954 ms 2024-02-21T23:38:30.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16562 0.13 MB/sec execute 537 sec latency 341.922 ms 2024-02-21T23:38:31.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16579 0.13 MB/sec execute 538 sec latency 678.856 ms 2024-02-21T23:38:32.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16596 0.13 MB/sec execute 539 sec latency 311.480 ms 2024-02-21T23:38:33.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16624 0.13 MB/sec execute 540 sec latency 309.033 ms 2024-02-21T23:38:34.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16631 0.13 MB/sec execute 541 sec latency 324.499 ms 2024-02-21T23:38:35.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16650 0.13 MB/sec execute 542 sec latency 151.708 ms 2024-02-21T23:38:36.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16685 0.13 MB/sec execute 543 sec latency 312.749 ms 2024-02-21T23:38:37.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16696 0.13 MB/sec execute 544 sec latency 326.128 ms 2024-02-21T23:38:38.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16746 0.13 MB/sec execute 545 sec latency 310.234 ms 2024-02-21T23:38:39.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16757 0.13 MB/sec execute 546 sec latency 332.475 ms 2024-02-21T23:38:40.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16776 0.13 MB/sec execute 547 sec latency 327.179 ms 2024-02-21T23:38:41.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16795 0.13 MB/sec execute 548 sec latency 383.372 ms 2024-02-21T23:38:42.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16811 0.13 MB/sec execute 549 sec latency 331.366 ms 2024-02-21T23:38:43.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16816 0.13 MB/sec execute 550 sec latency 329.775 ms 2024-02-21T23:38:44.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16939 0.13 MB/sec execute 551 sec latency 347.968 ms 2024-02-21T23:38:45.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16956 0.13 MB/sec execute 552 sec latency 338.874 ms 2024-02-21T23:38:46.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16974 0.13 MB/sec execute 553 sec latency 307.299 ms 2024-02-21T23:38:47.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17073 0.13 MB/sec execute 554 sec latency 308.184 ms 2024-02-21T23:38:48.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17118 0.13 MB/sec execute 555 sec latency 139.338 ms 2024-02-21T23:38:49.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17158 0.13 MB/sec execute 556 sec latency 308.720 ms 2024-02-21T23:38:50.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17167 0.13 MB/sec execute 557 sec latency 318.220 ms 2024-02-21T23:38:51.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17186 0.13 MB/sec execute 558 sec latency 288.418 ms 2024-02-21T23:38:52.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17216 0.13 MB/sec execute 559 sec latency 298.445 ms 2024-02-21T23:38:53.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17361 0.13 MB/sec execute 560 sec latency 355.193 ms 2024-02-21T23:38:54.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17382 0.13 MB/sec execute 561 sec latency 308.076 ms 2024-02-21T23:38:55.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17458 0.14 MB/sec execute 562 sec latency 209.859 ms 2024-02-21T23:38:56.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17472 0.14 MB/sec execute 563 sec latency 439.419 ms 2024-02-21T23:38:57.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17475 0.14 MB/sec execute 564 sec latency 312.816 ms 2024-02-21T23:38:58.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17479 0.14 MB/sec execute 565 sec latency 323.326 ms 2024-02-21T23:38:59.631 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:38:59.634 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:38:59.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17484 0.14 MB/sec execute 566 sec latency 381.263 ms 2024-02-21T23:39:00.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17487 0.13 MB/sec execute 567 sec latency 372.073 ms 2024-02-21T23:39:01.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17491 0.13 MB/sec execute 568 sec latency 431.065 ms 2024-02-21T23:39:02.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17499 0.13 MB/sec execute 569 sec latency 379.444 ms 2024-02-21T23:39:03.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17502 0.13 MB/sec execute 570 sec latency 383.081 ms 2024-02-21T23:39:04.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17505 0.13 MB/sec execute 571 sec latency 328.716 ms 2024-02-21T23:39:05.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17509 0.13 MB/sec execute 572 sec latency 336.869 ms 2024-02-21T23:39:06.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17512 0.13 MB/sec execute 573 sec latency 448.957 ms 2024-02-21T23:39:07.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17516 0.13 MB/sec execute 574 sec latency 313.225 ms 2024-02-21T23:39:08.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17519 0.13 MB/sec execute 575 sec latency 328.293 ms 2024-02-21T23:39:09.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17524 0.13 MB/sec execute 576 sec latency 311.836 ms 2024-02-21T23:39:10.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17528 0.13 MB/sec execute 577 sec latency 310.117 ms 2024-02-21T23:39:11.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17544 0.13 MB/sec execute 578 sec latency 322.108 ms 2024-02-21T23:39:12.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17566 0.13 MB/sec execute 579 sec latency 135.440 ms 2024-02-21T23:39:13.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17585 0.13 MB/sec execute 580 sec latency 307.060 ms 2024-02-21T23:39:14.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17685 0.14 MB/sec execute 581 sec latency 333.613 ms 2024-02-21T23:39:15.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17720 0.14 MB/sec execute 582 sec latency 240.609 ms 2024-02-21T23:39:16.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17746 0.14 MB/sec execute 583 sec latency 344.791 ms 2024-02-21T23:39:17.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17761 0.14 MB/sec execute 584 sec latency 337.182 ms 2024-02-21T23:39:18.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17811 0.14 MB/sec execute 585 sec latency 342.113 ms 2024-02-21T23:39:19.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17828 0.14 MB/sec execute 586 sec latency 335.664 ms 2024-02-21T23:39:20.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17856 0.14 MB/sec execute 587 sec latency 245.963 ms 2024-02-21T23:39:21.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17869 0.14 MB/sec execute 588 sec latency 495.185 ms 2024-02-21T23:39:22.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17898 0.14 MB/sec execute 589 sec latency 328.301 ms 2024-02-21T23:39:23.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17943 0.14 MB/sec execute 590 sec latency 312.977 ms 2024-02-21T23:39:24.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17963 0.14 MB/sec execute 591 sec latency 231.104 ms 2024-02-21T23:39:25.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17974 0.14 MB/sec execute 592 sec latency 205.159 ms 2024-02-21T23:39:26.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 17987 0.14 MB/sec execute 593 sec latency 220.483 ms 2024-02-21T23:39:27.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 18003 0.14 MB/sec execute 594 sec latency 145.645 ms 2024-02-21T23:39:28.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 18020 0.14 MB/sec execute 595 sec latency 161.489 ms 2024-02-21T23:39:29.741 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:39:29.744 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:39:29.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 18037 0.14 MB/sec execute 596 sec latency 163.478 ms 2024-02-21T23:39:30.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 18052 0.14 MB/sec execute 597 sec latency 173.804 ms 2024-02-21T23:39:31.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 18061 0.14 MB/sec execute 598 sec latency 407.163 ms 2024-02-21T23:39:32.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 18081 0.14 MB/sec execute 599 sec latency 325.310 ms 2024-02-21T23:39:33.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 600 sec 2024-02-21T23:39:34.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 601 sec 2024-02-21T23:39:35.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 602 sec 2024-02-21T23:39:36.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 603 sec 2024-02-21T23:39:37.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 604 sec 2024-02-21T23:39:38.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 605 sec 2024-02-21T23:39:39.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 606 sec 2024-02-21T23:39:40.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 607 sec 2024-02-21T23:39:41.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 608 sec 2024-02-21T23:39:42.791 INFO:tasks.workunit.client.0.smithi133.stdout: 0 cleanup 609 sec 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: Operation Count AvgLat MaxLat 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: ---------------------------------------- 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: NTCreateX 2576 63.349 280.992 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: Close 1882 1.153 12.991 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: Rename 108 179.075 465.192 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: Unlink 525 69.774 186.535 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: Qpathinfo 2316 34.663 254.550 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: Qfileinfo 396 0.933 1.492 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: Qfsinfo 435 7.858 141.668 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: Sfileinfo 200 11.471 16.861 2024-02-21T23:39:42.792 INFO:tasks.workunit.client.0.smithi133.stdout: Find 877 318.134 678.847 2024-02-21T23:39:42.793 INFO:tasks.workunit.client.0.smithi133.stdout: WriteX 1254 3.961 97.129 2024-02-21T23:39:42.793 INFO:tasks.workunit.client.0.smithi133.stdout: ReadX 3999 0.023 2.260 2024-02-21T23:39:42.793 INFO:tasks.workunit.client.0.smithi133.stdout: LockX 8 15.521 17.866 2024-02-21T23:39:42.793 INFO:tasks.workunit.client.0.smithi133.stdout: UnlockX 8 15.537 17.263 2024-02-21T23:39:42.793 INFO:tasks.workunit.client.0.smithi133.stdout: Flush 172 46.555 146.559 2024-02-21T23:39:42.793 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-02-21T23:39:42.793 INFO:tasks.workunit.client.0.smithi133.stdout:Throughput 0.135475 MB/sec 1 clients 1 procs max_latency=678.856 ms 2024-02-21T23:39:42.793 INFO:tasks.workunit.client.0.smithi133.stderr:+ dbench 10 2024-02-21T23:39:42.794 INFO:tasks.workunit.client.0.smithi133.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-02-21T23:39:42.794 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-02-21T23:39:42.794 INFO:tasks.workunit.client.0.smithi133.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-02-21T23:39:42.794 INFO:tasks.workunit.client.0.smithi133.stdout:8 of 10 processes prepared for launch 0 sec 2024-02-21T23:39:42.893 INFO:tasks.workunit.client.0.smithi133.stdout:10 of 10 processes prepared for launch 0 sec 2024-02-21T23:39:42.893 INFO:tasks.workunit.client.0.smithi133.stdout:releasing clients 2024-02-21T23:39:43.893 INFO:tasks.workunit.client.0.smithi133.stdout: 0 2 0.00 MB/sec warmup 1 sec latency 999.912 ms 2024-02-21T23:39:44.893 INFO:tasks.workunit.client.0.smithi133.stdout: 0 6 0.00 MB/sec warmup 2 sec latency 1999.955 ms 2024-02-21T23:39:45.893 INFO:tasks.workunit.client.0.smithi133.stdout: 2 11 0.28 MB/sec warmup 3 sec latency 662.800 ms 2024-02-21T23:39:46.893 INFO:tasks.workunit.client.0.smithi133.stdout: 8 27 2.13 MB/sec warmup 4 sec latency 724.955 ms 2024-02-21T23:39:47.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 45 3.53 MB/sec warmup 5 sec latency 772.455 ms 2024-02-21T23:39:48.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 62 4.47 MB/sec warmup 6 sec latency 639.478 ms 2024-02-21T23:39:49.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 79 5.03 MB/sec warmup 7 sec latency 675.539 ms 2024-02-21T23:39:50.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 95 5.51 MB/sec warmup 8 sec latency 708.854 ms 2024-02-21T23:39:51.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 109 5.63 MB/sec warmup 9 sec latency 807.305 ms 2024-02-21T23:39:52.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 117 5.40 MB/sec warmup 10 sec latency 934.649 ms 2024-02-21T23:39:53.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 125 5.14 MB/sec warmup 11 sec latency 986.476 ms 2024-02-21T23:39:54.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 132 4.88 MB/sec warmup 12 sec latency 1195.346 ms 2024-02-21T23:39:55.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 138 4.63 MB/sec warmup 13 sec latency 1381.571 ms 2024-02-21T23:39:56.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 143 4.41 MB/sec warmup 14 sec latency 1017.345 ms 2024-02-21T23:39:57.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 152 4.29 MB/sec warmup 15 sec latency 1253.037 ms 2024-02-21T23:39:58.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 156 4.10 MB/sec warmup 16 sec latency 1588.349 ms 2024-02-21T23:39:59.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 160 3.93 MB/sec warmup 17 sec latency 1034.705 ms 2024-02-21T23:39:59.906 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:39:59.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:40:00.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 166 3.79 MB/sec warmup 18 sec latency 1562.902 ms 2024-02-21T23:40:01.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 171 3.69 MB/sec warmup 19 sec latency 1342.969 ms 2024-02-21T23:40:02.893 INFO:tasks.workunit.client.0.smithi133.stdout: 10 173 3.54 MB/sec warmup 20 sec latency 1954.854 ms 2024-02-21T23:40:03.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 179 3.46 MB/sec warmup 21 sec latency 2112.179 ms 2024-02-21T23:40:04.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 188 3.44 MB/sec warmup 22 sec latency 1738.022 ms 2024-02-21T23:40:05.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 195 3.39 MB/sec warmup 23 sec latency 1225.152 ms 2024-02-21T23:40:06.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 199 3.29 MB/sec warmup 24 sec latency 1358.181 ms 2024-02-21T23:40:07.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 204 3.21 MB/sec warmup 25 sec latency 1322.284 ms 2024-02-21T23:40:08.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 211 3.17 MB/sec warmup 26 sec latency 1167.153 ms 2024-02-21T23:40:09.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 218 3.14 MB/sec warmup 27 sec latency 1258.612 ms 2024-02-21T23:40:10.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 225 3.11 MB/sec warmup 28 sec latency 1179.900 ms 2024-02-21T23:40:11.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 228 3.04 MB/sec warmup 29 sec latency 1607.638 ms 2024-02-21T23:40:12.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 235 3.00 MB/sec warmup 30 sec latency 1781.732 ms 2024-02-21T23:40:13.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 243 2.99 MB/sec warmup 31 sec latency 1307.227 ms 2024-02-21T23:40:14.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 245 2.92 MB/sec warmup 32 sec latency 1467.160 ms 2024-02-21T23:40:15.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 252 2.90 MB/sec warmup 33 sec latency 1709.694 ms 2024-02-21T23:40:16.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 258 2.87 MB/sec warmup 34 sec latency 1530.392 ms 2024-02-21T23:40:17.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 264 2.85 MB/sec warmup 35 sec latency 1598.453 ms 2024-02-21T23:40:18.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 269 2.81 MB/sec warmup 36 sec latency 1754.167 ms 2024-02-21T23:40:19.199 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:15:47.107 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:40:19.253 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:15:47.161 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:40:19.259 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:15:47.169 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:40:19.260 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:15:47.170 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:40:19.260 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:15:47.171 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:40:19.260 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:15:47.172 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:40:19.339 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:15:47.247 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:40:19.345 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:15:47.258 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:40:19.349 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:15:47.262 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:40:19.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 275 2.79 MB/sec warmup 37 sec latency 1797.385 ms 2024-02-21T23:40:20.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 280 2.76 MB/sec warmup 38 sec latency 1745.815 ms 2024-02-21T23:40:21.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 286 2.74 MB/sec warmup 39 sec latency 1583.395 ms 2024-02-21T23:40:22.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 289 2.69 MB/sec warmup 40 sec latency 1883.809 ms 2024-02-21T23:40:23.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 293 2.65 MB/sec warmup 41 sec latency 2594.013 ms 2024-02-21T23:40:24.894 INFO:tasks.workunit.client.0.smithi133.stdout: 10 300 2.63 MB/sec warmup 42 sec latency 2093.113 ms 2024-02-21T23:40:25.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 306 2.61 MB/sec warmup 43 sec latency 1094.208 ms 2024-02-21T23:40:26.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 313 2.61 MB/sec warmup 44 sec latency 1061.597 ms 2024-02-21T23:40:27.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 320 2.60 MB/sec warmup 45 sec latency 1198.476 ms 2024-02-21T23:40:28.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 325 2.57 MB/sec warmup 46 sec latency 1333.604 ms 2024-02-21T23:40:29.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 333 2.57 MB/sec warmup 47 sec latency 1124.880 ms 2024-02-21T23:40:29.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:40:29.967 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:40:30.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 337 2.54 MB/sec warmup 48 sec latency 1326.104 ms 2024-02-21T23:40:31.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 343 2.52 MB/sec warmup 49 sec latency 1378.958 ms 2024-02-21T23:40:32.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 346 2.49 MB/sec warmup 50 sec latency 1836.461 ms 2024-02-21T23:40:33.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 351 2.48 MB/sec warmup 51 sec latency 2265.074 ms 2024-02-21T23:40:34.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 356 2.45 MB/sec warmup 52 sec latency 1492.847 ms 2024-02-21T23:40:35.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 364 2.46 MB/sec warmup 53 sec latency 1333.459 ms 2024-02-21T23:40:36.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 370 2.45 MB/sec warmup 54 sec latency 1409.846 ms 2024-02-21T23:40:37.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 378 2.45 MB/sec warmup 55 sec latency 1474.663 ms 2024-02-21T23:40:38.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 384 2.44 MB/sec warmup 56 sec latency 1191.209 ms 2024-02-21T23:40:39.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 390 2.43 MB/sec warmup 57 sec latency 1340.334 ms 2024-02-21T23:40:40.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 397 2.43 MB/sec warmup 58 sec latency 1510.631 ms 2024-02-21T23:40:41.895 INFO:tasks.workunit.client.0.smithi133.stdout: 10 403 2.43 MB/sec warmup 59 sec latency 1438.790 ms 2024-02-21T23:40:42.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 405 2.39 MB/sec warmup 60 sec latency 2168.442 ms 2024-02-21T23:40:43.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 411 2.38 MB/sec warmup 61 sec latency 2255.105 ms 2024-02-21T23:40:44.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 413 2.36 MB/sec warmup 62 sec latency 1728.439 ms 2024-02-21T23:40:45.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 421 2.36 MB/sec warmup 63 sec latency 1736.677 ms 2024-02-21T23:40:46.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 427 2.36 MB/sec warmup 64 sec latency 1544.832 ms 2024-02-21T23:40:47.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 433 2.35 MB/sec warmup 65 sec latency 1631.279 ms 2024-02-21T23:40:48.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 437 2.33 MB/sec warmup 66 sec latency 1483.977 ms 2024-02-21T23:40:49.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 445 2.34 MB/sec warmup 67 sec latency 1701.580 ms 2024-02-21T23:40:50.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 447 2.31 MB/sec warmup 68 sec latency 1455.722 ms 2024-02-21T23:40:51.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 453 2.31 MB/sec warmup 69 sec latency 2076.597 ms 2024-02-21T23:40:52.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 458 2.29 MB/sec warmup 70 sec latency 1526.846 ms 2024-02-21T23:40:53.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 460 2.27 MB/sec warmup 71 sec latency 2235.202 ms 2024-02-21T23:40:54.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 468 2.28 MB/sec warmup 72 sec latency 2584.400 ms 2024-02-21T23:40:55.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 473 2.27 MB/sec warmup 73 sec latency 1665.293 ms 2024-02-21T23:40:56.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 485 2.30 MB/sec warmup 74 sec latency 1629.651 ms 2024-02-21T23:40:57.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 489 2.28 MB/sec warmup 75 sec latency 1697.880 ms 2024-02-21T23:40:58.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 493 2.27 MB/sec warmup 76 sec latency 2532.100 ms 2024-02-21T23:40:59.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 500 2.27 MB/sec warmup 77 sec latency 3532.240 ms 2024-02-21T23:41:00.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:41:00.071 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:41:00.896 INFO:tasks.workunit.client.0.smithi133.stdout: 10 506 2.26 MB/sec warmup 78 sec latency 4120.208 ms 2024-02-21T23:41:01.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 509 2.24 MB/sec warmup 79 sec latency 1649.794 ms 2024-02-21T23:41:02.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 510 2.22 MB/sec warmup 80 sec latency 2110.384 ms 2024-02-21T23:41:03.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 514 2.21 MB/sec warmup 81 sec latency 2269.685 ms 2024-02-21T23:41:04.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 518 2.19 MB/sec warmup 82 sec latency 2993.487 ms 2024-02-21T23:41:05.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 525 2.20 MB/sec warmup 83 sec latency 3993.531 ms 2024-02-21T23:41:06.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 535 2.21 MB/sec warmup 84 sec latency 4412.098 ms 2024-02-21T23:41:07.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 550 2.25 MB/sec warmup 85 sec latency 1657.804 ms 2024-02-21T23:41:08.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 563 2.26 MB/sec warmup 86 sec latency 2346.964 ms 2024-02-21T23:41:09.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 586 2.31 MB/sec warmup 87 sec latency 2572.791 ms 2024-02-21T23:41:10.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 604 2.36 MB/sec warmup 88 sec latency 1771.402 ms 2024-02-21T23:41:11.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 609 2.36 MB/sec warmup 89 sec latency 2129.427 ms 2024-02-21T23:41:12.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 610 2.33 MB/sec warmup 90 sec latency 3094.077 ms 2024-02-21T23:41:13.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 612 2.31 MB/sec warmup 91 sec latency 3064.207 ms 2024-02-21T23:41:14.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 616 2.28 MB/sec warmup 92 sec latency 3539.744 ms 2024-02-21T23:41:15.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 621 2.26 MB/sec warmup 93 sec latency 3130.919 ms 2024-02-21T23:41:16.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 622 2.23 MB/sec warmup 94 sec latency 2519.846 ms 2024-02-21T23:41:17.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 624 2.21 MB/sec warmup 95 sec latency 2692.313 ms 2024-02-21T23:41:18.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 636 2.19 MB/sec warmup 96 sec latency 2487.814 ms 2024-02-21T23:41:19.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 645 2.17 MB/sec warmup 97 sec latency 2224.553 ms 2024-02-21T23:41:20.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 648 2.15 MB/sec warmup 98 sec latency 2628.285 ms 2024-02-21T23:41:21.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 649 2.13 MB/sec warmup 99 sec latency 2652.153 ms 2024-02-21T23:41:22.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 650 2.11 MB/sec warmup 100 sec latency 2656.366 ms 2024-02-21T23:41:23.897 INFO:tasks.workunit.client.0.smithi133.stdout: 10 651 2.09 MB/sec warmup 101 sec latency 2628.754 ms 2024-02-21T23:41:24.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 656 2.07 MB/sec warmup 102 sec latency 2552.043 ms 2024-02-21T23:41:25.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 659 2.05 MB/sec warmup 103 sec latency 2052.811 ms 2024-02-21T23:41:26.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 662 2.03 MB/sec warmup 104 sec latency 2754.513 ms 2024-02-21T23:41:27.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 664 2.01 MB/sec warmup 105 sec latency 2889.918 ms 2024-02-21T23:41:28.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 668 1.99 MB/sec warmup 106 sec latency 2479.478 ms 2024-02-21T23:41:29.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 676 1.98 MB/sec warmup 107 sec latency 2684.909 ms 2024-02-21T23:41:30.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:41:30.356 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:41:30.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 685 1.96 MB/sec warmup 108 sec latency 2391.074 ms 2024-02-21T23:41:31.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 690 1.95 MB/sec warmup 109 sec latency 1980.518 ms 2024-02-21T23:41:32.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 691 1.93 MB/sec warmup 110 sec latency 2318.259 ms 2024-02-21T23:41:33.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 693 1.91 MB/sec warmup 111 sec latency 2909.431 ms 2024-02-21T23:41:34.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 695 1.89 MB/sec warmup 112 sec latency 3038.381 ms 2024-02-21T23:41:35.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 700 1.88 MB/sec warmup 113 sec latency 2461.867 ms 2024-02-21T23:41:36.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 710 1.86 MB/sec warmup 114 sec latency 2276.385 ms 2024-02-21T23:41:37.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 719 1.85 MB/sec warmup 115 sec latency 1309.377 ms 2024-02-21T23:41:38.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 722 1.83 MB/sec warmup 116 sec latency 2309.403 ms 2024-02-21T23:41:39.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 724 1.82 MB/sec warmup 117 sec latency 3309.431 ms 2024-02-21T23:41:40.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 726 1.80 MB/sec warmup 118 sec latency 3930.424 ms 2024-02-21T23:41:41.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 732 1.79 MB/sec warmup 119 sec latency 4080.133 ms 2024-02-21T23:41:43.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 745 0.08 MB/sec execute 1 sec latency 3346.369 ms 2024-02-21T23:41:44.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 752 0.08 MB/sec execute 2 sec latency 3406.502 ms 2024-02-21T23:41:45.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 756 0.07 MB/sec execute 3 sec latency 3608.376 ms 2024-02-21T23:41:46.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 757 0.05 MB/sec execute 4 sec latency 3527.012 ms 2024-02-21T23:41:47.898 INFO:tasks.workunit.client.0.smithi133.stdout: 10 760 0.06 MB/sec execute 5 sec latency 3553.833 ms 2024-02-21T23:41:48.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 763 0.07 MB/sec execute 6 sec latency 2927.167 ms 2024-02-21T23:41:49.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 769 0.07 MB/sec execute 7 sec latency 2275.073 ms 2024-02-21T23:41:50.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 781 0.09 MB/sec execute 8 sec latency 2392.878 ms 2024-02-21T23:41:51.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 785 0.08 MB/sec execute 9 sec latency 2495.192 ms 2024-02-21T23:41:52.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 789 0.08 MB/sec execute 10 sec latency 2273.359 ms 2024-02-21T23:41:53.156 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:17:21.065 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:41:53.205 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:17:21.114 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:41:53.236 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:17:21.145 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:41:53.259 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:17:21.172 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:41:53.260 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:17:21.173 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:41:53.260 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:17:21.174 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:41:53.262 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:17:21.175 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:41:53.295 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:17:21.209 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:41:53.316 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:17:21.229 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:41:53.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 789 0.07 MB/sec execute 11 sec latency 3273.489 ms 2024-02-21T23:41:54.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 794 0.09 MB/sec execute 12 sec latency 4273.519 ms 2024-02-21T23:41:55.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 802 0.11 MB/sec execute 13 sec latency 4351.595 ms 2024-02-21T23:41:56.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 809 0.11 MB/sec execute 14 sec latency 3948.091 ms 2024-02-21T23:41:57.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 813 0.11 MB/sec execute 15 sec latency 2733.156 ms 2024-02-21T23:41:58.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 814 0.10 MB/sec execute 16 sec latency 2822.295 ms 2024-02-21T23:41:59.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 818 0.10 MB/sec execute 17 sec latency 2613.336 ms 2024-02-21T23:42:00.467 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:42:00.470 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:42:00.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 819 0.09 MB/sec execute 18 sec latency 2353.965 ms 2024-02-21T23:42:01.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 823 0.09 MB/sec execute 19 sec latency 2190.713 ms 2024-02-21T23:42:02.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 824 0.09 MB/sec execute 20 sec latency 2507.297 ms 2024-02-21T23:42:03.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 828 0.09 MB/sec execute 21 sec latency 2610.392 ms 2024-02-21T23:42:04.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 831 0.09 MB/sec execute 22 sec latency 2623.826 ms 2024-02-21T23:42:05.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 832 0.08 MB/sec execute 23 sec latency 2349.820 ms 2024-02-21T23:42:06.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 841 0.09 MB/sec execute 24 sec latency 2081.011 ms 2024-02-21T23:42:07.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 844 0.09 MB/sec execute 25 sec latency 2444.811 ms 2024-02-21T23:42:08.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 857 0.10 MB/sec execute 26 sec latency 2484.185 ms 2024-02-21T23:42:09.899 INFO:tasks.workunit.client.0.smithi133.stdout: 10 857 0.10 MB/sec execute 27 sec latency 2604.045 ms 2024-02-21T23:42:10.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 860 0.10 MB/sec execute 28 sec latency 2270.157 ms 2024-02-21T23:42:11.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 863 0.09 MB/sec execute 29 sec latency 2434.618 ms 2024-02-21T23:42:12.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 865 0.09 MB/sec execute 30 sec latency 2380.119 ms 2024-02-21T23:42:13.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 867 0.09 MB/sec execute 31 sec latency 2774.206 ms 2024-02-21T23:42:14.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 872 0.09 MB/sec execute 32 sec latency 2846.715 ms 2024-02-21T23:42:15.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 874 0.09 MB/sec execute 33 sec latency 3067.611 ms 2024-02-21T23:42:16.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 878 0.09 MB/sec execute 34 sec latency 2876.099 ms 2024-02-21T23:42:17.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 882 0.09 MB/sec execute 35 sec latency 3178.966 ms 2024-02-21T23:42:18.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 886 0.09 MB/sec execute 36 sec latency 3236.151 ms 2024-02-21T23:42:19.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 892 0.10 MB/sec execute 37 sec latency 2698.723 ms 2024-02-21T23:42:20.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 897 0.10 MB/sec execute 38 sec latency 2997.018 ms 2024-02-21T23:42:21.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 899 0.10 MB/sec execute 39 sec latency 2855.946 ms 2024-02-21T23:42:22.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 904 0.10 MB/sec execute 40 sec latency 2977.404 ms 2024-02-21T23:42:23.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 908 0.10 MB/sec execute 41 sec latency 3029.011 ms 2024-02-21T23:42:24.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 912 0.09 MB/sec execute 42 sec latency 3808.811 ms 2024-02-21T23:42:25.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 916 0.10 MB/sec execute 43 sec latency 3930.426 ms 2024-02-21T23:42:26.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 921 0.10 MB/sec execute 44 sec latency 3960.412 ms 2024-02-21T23:42:27.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 925 0.10 MB/sec execute 45 sec latency 3138.331 ms 2024-02-21T23:42:28.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 928 0.10 MB/sec execute 46 sec latency 3671.355 ms 2024-02-21T23:42:29.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 928 0.10 MB/sec execute 47 sec latency 3468.269 ms 2024-02-21T23:42:30.555 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:42:30.558 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:42:30.900 INFO:tasks.workunit.client.0.smithi133.stdout: 10 929 0.09 MB/sec execute 48 sec latency 3301.603 ms 2024-02-21T23:42:31.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 930 0.09 MB/sec execute 49 sec latency 2269.688 ms 2024-02-21T23:42:32.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 931 0.09 MB/sec execute 50 sec latency 2358.370 ms 2024-02-21T23:42:33.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 932 0.09 MB/sec execute 51 sec latency 2480.794 ms 2024-02-21T23:42:34.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 935 0.09 MB/sec execute 52 sec latency 2345.966 ms 2024-02-21T23:42:35.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 936 0.09 MB/sec execute 53 sec latency 2133.321 ms 2024-02-21T23:42:36.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 940 0.09 MB/sec execute 54 sec latency 2239.643 ms 2024-02-21T23:42:37.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 944 0.09 MB/sec execute 55 sec latency 2327.123 ms 2024-02-21T23:42:38.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 947 0.08 MB/sec execute 56 sec latency 2673.116 ms 2024-02-21T23:42:39.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 950 0.08 MB/sec execute 57 sec latency 2721.193 ms 2024-02-21T23:42:40.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 953 0.09 MB/sec execute 58 sec latency 2940.579 ms 2024-02-21T23:42:41.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 961 0.10 MB/sec execute 59 sec latency 2523.326 ms 2024-02-21T23:42:42.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 963 0.10 MB/sec execute 60 sec latency 2636.850 ms 2024-02-21T23:42:43.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 973 0.11 MB/sec execute 61 sec latency 2712.814 ms 2024-02-21T23:42:44.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 981 0.12 MB/sec execute 62 sec latency 2225.363 ms 2024-02-21T23:42:45.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 987 0.12 MB/sec execute 63 sec latency 1961.925 ms 2024-02-21T23:42:46.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 990 0.13 MB/sec execute 64 sec latency 2961.965 ms 2024-02-21T23:42:47.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 994 0.13 MB/sec execute 65 sec latency 3076.436 ms 2024-02-21T23:42:48.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1001 0.13 MB/sec execute 66 sec latency 3841.961 ms 2024-02-21T23:42:49.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1005 0.13 MB/sec execute 67 sec latency 3579.834 ms 2024-02-21T23:42:50.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1010 0.14 MB/sec execute 68 sec latency 3043.439 ms 2024-02-21T23:42:51.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1011 0.14 MB/sec execute 69 sec latency 2888.143 ms 2024-02-21T23:42:52.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1016 0.16 MB/sec execute 70 sec latency 2962.009 ms 2024-02-21T23:42:53.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1027 0.20 MB/sec execute 71 sec latency 3256.363 ms 2024-02-21T23:42:54.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1032 0.22 MB/sec execute 72 sec latency 2781.273 ms 2024-02-21T23:42:55.901 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1034 0.21 MB/sec execute 73 sec latency 2595.484 ms 2024-02-21T23:42:56.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1037 0.21 MB/sec execute 74 sec latency 2178.597 ms 2024-02-21T23:42:57.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1038 0.21 MB/sec execute 75 sec latency 2240.606 ms 2024-02-21T23:42:58.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1041 0.21 MB/sec execute 76 sec latency 2253.080 ms 2024-02-21T23:42:59.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1043 0.20 MB/sec execute 77 sec latency 2898.314 ms 2024-02-21T23:43:00.772 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:43:00.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:43:00.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1046 0.20 MB/sec execute 78 sec latency 2887.166 ms 2024-02-21T23:43:01.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1051 0.20 MB/sec execute 79 sec latency 2328.908 ms 2024-02-21T23:43:02.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1052 0.20 MB/sec execute 80 sec latency 2780.095 ms 2024-02-21T23:43:03.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1054 0.19 MB/sec execute 81 sec latency 2977.112 ms 2024-02-21T23:43:04.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1056 0.19 MB/sec execute 82 sec latency 2998.543 ms 2024-02-21T23:43:05.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1059 0.19 MB/sec execute 83 sec latency 2709.001 ms 2024-02-21T23:43:06.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1061 0.19 MB/sec execute 84 sec latency 2697.115 ms 2024-02-21T23:43:07.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1063 0.19 MB/sec execute 85 sec latency 2935.707 ms 2024-02-21T23:43:08.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1066 0.18 MB/sec execute 86 sec latency 2776.259 ms 2024-02-21T23:43:09.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1069 0.18 MB/sec execute 87 sec latency 3323.940 ms 2024-02-21T23:43:10.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1074 0.18 MB/sec execute 88 sec latency 3272.092 ms 2024-02-21T23:43:11.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1080 0.18 MB/sec execute 89 sec latency 2849.455 ms 2024-02-21T23:43:12.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1085 0.18 MB/sec execute 90 sec latency 2865.420 ms 2024-02-21T23:43:13.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1089 0.18 MB/sec execute 91 sec latency 2601.104 ms 2024-02-21T23:43:14.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1096 0.18 MB/sec execute 92 sec latency 3409.138 ms 2024-02-21T23:43:15.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1100 0.17 MB/sec execute 93 sec latency 3206.682 ms 2024-02-21T23:43:16.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1102 0.17 MB/sec execute 94 sec latency 3590.407 ms 2024-02-21T23:43:17.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1104 0.17 MB/sec execute 95 sec latency 2622.483 ms 2024-02-21T23:43:18.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1108 0.17 MB/sec execute 96 sec latency 3361.669 ms 2024-02-21T23:43:19.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1113 0.17 MB/sec execute 97 sec latency 1846.593 ms 2024-02-21T23:43:20.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1115 0.17 MB/sec execute 98 sec latency 2836.377 ms 2024-02-21T23:43:21.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1121 0.17 MB/sec execute 99 sec latency 2685.257 ms 2024-02-21T23:43:22.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1124 0.17 MB/sec execute 100 sec latency 2811.449 ms 2024-02-21T23:43:23.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1129 0.16 MB/sec execute 101 sec latency 2722.005 ms 2024-02-21T23:43:24.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1131 0.16 MB/sec execute 102 sec latency 2746.472 ms 2024-02-21T23:43:25.902 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1136 0.16 MB/sec execute 103 sec latency 2337.614 ms 2024-02-21T23:43:26.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1142 0.16 MB/sec execute 104 sec latency 2159.477 ms 2024-02-21T23:43:27.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1148 0.16 MB/sec execute 105 sec latency 1975.840 ms 2024-02-21T23:43:28.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1152 0.16 MB/sec execute 106 sec latency 2140.911 ms 2024-02-21T23:43:29.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1157 0.16 MB/sec execute 107 sec latency 2954.510 ms 2024-02-21T23:43:30.844 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:43:30.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:43:30.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1160 0.16 MB/sec execute 108 sec latency 1057.331 ms 2024-02-21T23:43:31.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1162 0.15 MB/sec execute 109 sec latency 2057.371 ms 2024-02-21T23:43:32.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1163 0.15 MB/sec execute 110 sec latency 2966.139 ms 2024-02-21T23:43:33.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1168 0.15 MB/sec execute 111 sec latency 3079.903 ms 2024-02-21T23:43:34.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1170 0.15 MB/sec execute 112 sec latency 3029.468 ms 2024-02-21T23:43:35.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1172 0.15 MB/sec execute 113 sec latency 3193.685 ms 2024-02-21T23:43:36.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1176 0.15 MB/sec execute 114 sec latency 2634.952 ms 2024-02-21T23:43:37.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1178 0.15 MB/sec execute 115 sec latency 2606.955 ms 2024-02-21T23:43:38.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1180 0.15 MB/sec execute 116 sec latency 2602.134 ms 2024-02-21T23:43:39.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1184 0.15 MB/sec execute 117 sec latency 2887.338 ms 2024-02-21T23:43:40.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1185 0.14 MB/sec execute 118 sec latency 2587.073 ms 2024-02-21T23:43:41.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1187 0.14 MB/sec execute 119 sec latency 2719.878 ms 2024-02-21T23:43:42.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1189 0.14 MB/sec execute 120 sec latency 3018.198 ms 2024-02-21T23:43:43.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1195 0.14 MB/sec execute 121 sec latency 2549.569 ms 2024-02-21T23:43:44.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1196 0.14 MB/sec execute 122 sec latency 2702.613 ms 2024-02-21T23:43:45.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1202 0.14 MB/sec execute 123 sec latency 3386.405 ms 2024-02-21T23:43:46.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1205 0.14 MB/sec execute 124 sec latency 1998.114 ms 2024-02-21T23:43:47.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1213 0.14 MB/sec execute 125 sec latency 2679.213 ms 2024-02-21T23:43:48.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1216 0.14 MB/sec execute 126 sec latency 2970.120 ms 2024-02-21T23:43:49.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1222 0.14 MB/sec execute 127 sec latency 3131.754 ms 2024-02-21T23:43:50.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1226 0.14 MB/sec execute 128 sec latency 2654.208 ms 2024-02-21T23:43:51.903 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1230 0.14 MB/sec execute 129 sec latency 2860.841 ms 2024-02-21T23:43:52.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1233 0.14 MB/sec execute 130 sec latency 3001.177 ms 2024-02-21T23:43:53.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1236 0.13 MB/sec execute 131 sec latency 3248.979 ms 2024-02-21T23:43:54.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1241 0.13 MB/sec execute 132 sec latency 3591.606 ms 2024-02-21T23:43:55.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1247 0.13 MB/sec execute 133 sec latency 3105.622 ms 2024-02-21T23:43:56.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1252 0.13 MB/sec execute 134 sec latency 2643.884 ms 2024-02-21T23:43:57.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1256 0.13 MB/sec execute 135 sec latency 2675.510 ms 2024-02-21T23:43:58.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1262 0.14 MB/sec execute 136 sec latency 2860.577 ms 2024-02-21T23:43:59.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1270 0.15 MB/sec execute 137 sec latency 2968.539 ms 2024-02-21T23:44:00.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1273 0.15 MB/sec execute 138 sec latency 3081.087 ms 2024-02-21T23:44:00.957 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:44:00.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:44:01.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1277 0.15 MB/sec execute 139 sec latency 2779.807 ms 2024-02-21T23:44:02.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1282 0.15 MB/sec execute 140 sec latency 2916.086 ms 2024-02-21T23:44:03.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1283 0.15 MB/sec execute 141 sec latency 3791.381 ms 2024-02-21T23:44:04.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1294 0.17 MB/sec execute 142 sec latency 4024.516 ms 2024-02-21T23:44:05.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1298 0.17 MB/sec execute 143 sec latency 3887.981 ms 2024-02-21T23:44:06.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1299 0.17 MB/sec execute 144 sec latency 3716.634 ms 2024-02-21T23:44:07.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1303 0.17 MB/sec execute 145 sec latency 2872.915 ms 2024-02-21T23:44:08.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1310 0.18 MB/sec execute 146 sec latency 3048.301 ms 2024-02-21T23:44:09.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1310 0.18 MB/sec execute 147 sec latency 2417.560 ms 2024-02-21T23:44:10.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1321 0.20 MB/sec execute 148 sec latency 2225.726 ms 2024-02-21T23:44:11.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1322 0.20 MB/sec execute 149 sec latency 2311.014 ms 2024-02-21T23:44:12.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1323 0.20 MB/sec execute 150 sec latency 2307.589 ms 2024-02-21T23:44:13.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1337 0.22 MB/sec execute 151 sec latency 2621.268 ms 2024-02-21T23:44:14.904 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1361 0.27 MB/sec execute 152 sec latency 2436.653 ms 2024-02-21T23:44:15.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1372 0.29 MB/sec execute 153 sec latency 2473.660 ms 2024-02-21T23:44:16.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1384 0.30 MB/sec execute 154 sec latency 2533.310 ms 2024-02-21T23:44:17.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1384 0.30 MB/sec execute 155 sec latency 2001.808 ms 2024-02-21T23:44:18.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1386 0.30 MB/sec execute 156 sec latency 2245.908 ms 2024-02-21T23:44:19.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1390 0.30 MB/sec execute 157 sec latency 2252.025 ms 2024-02-21T23:44:20.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1392 0.30 MB/sec execute 158 sec latency 2291.245 ms 2024-02-21T23:44:21.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1394 0.29 MB/sec execute 159 sec latency 2520.276 ms 2024-02-21T23:44:22.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1396 0.29 MB/sec execute 160 sec latency 2522.634 ms 2024-02-21T23:44:23.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1400 0.29 MB/sec execute 161 sec latency 2494.107 ms 2024-02-21T23:44:24.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1405 0.29 MB/sec execute 162 sec latency 2485.401 ms 2024-02-21T23:44:25.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1408 0.29 MB/sec execute 163 sec latency 2638.545 ms 2024-02-21T23:44:26.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1411 0.29 MB/sec execute 164 sec latency 2161.659 ms 2024-02-21T23:44:27.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1413 0.28 MB/sec execute 165 sec latency 2494.122 ms 2024-02-21T23:44:28.142 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:19:56.052 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:44:28.148 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:19:56.058 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:44:28.201 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:19:56.115 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:44:28.203 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:19:56.117 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:44:28.203 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:19:56.117 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:44:28.206 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:19:56.120 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:44:28.215 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:19:56.124 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:44:28.268 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:19:56.182 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:44:28.268 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:19:56.182 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:44:28.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1418 0.28 MB/sec execute 166 sec latency 2601.136 ms 2024-02-21T23:44:29.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1422 0.28 MB/sec execute 167 sec latency 2762.329 ms 2024-02-21T23:44:30.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1428 0.28 MB/sec execute 168 sec latency 2051.538 ms 2024-02-21T23:44:31.079 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:44:31.081 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:44:31.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1433 0.28 MB/sec execute 169 sec latency 2245.608 ms 2024-02-21T23:44:32.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1437 0.28 MB/sec execute 170 sec latency 2675.801 ms 2024-02-21T23:44:33.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1441 0.27 MB/sec execute 171 sec latency 2978.298 ms 2024-02-21T23:44:34.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1444 0.27 MB/sec execute 172 sec latency 3007.715 ms 2024-02-21T23:44:35.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1449 0.27 MB/sec execute 173 sec latency 2675.857 ms 2024-02-21T23:44:36.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1453 0.27 MB/sec execute 174 sec latency 2531.022 ms 2024-02-21T23:44:37.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1459 0.27 MB/sec execute 175 sec latency 2741.172 ms 2024-02-21T23:44:38.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1463 0.27 MB/sec execute 176 sec latency 1805.299 ms 2024-02-21T23:44:39.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1465 0.27 MB/sec execute 177 sec latency 2525.902 ms 2024-02-21T23:44:40.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1469 0.26 MB/sec execute 178 sec latency 2888.883 ms 2024-02-21T23:44:41.905 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1475 0.26 MB/sec execute 179 sec latency 2387.845 ms 2024-02-21T23:44:42.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1480 0.26 MB/sec execute 180 sec latency 2058.740 ms 2024-02-21T23:44:43.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1482 0.26 MB/sec execute 181 sec latency 2452.147 ms 2024-02-21T23:44:44.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1485 0.26 MB/sec execute 182 sec latency 2427.780 ms 2024-02-21T23:44:45.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1489 0.26 MB/sec execute 183 sec latency 2650.152 ms 2024-02-21T23:44:46.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1492 0.26 MB/sec execute 184 sec latency 2425.516 ms 2024-02-21T23:44:47.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1499 0.26 MB/sec execute 185 sec latency 2116.993 ms 2024-02-21T23:44:48.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1501 0.25 MB/sec execute 186 sec latency 2892.693 ms 2024-02-21T23:44:49.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1503 0.25 MB/sec execute 187 sec latency 2895.170 ms 2024-02-21T23:44:50.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1505 0.25 MB/sec execute 188 sec latency 2808.022 ms 2024-02-21T23:44:51.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1510 0.25 MB/sec execute 189 sec latency 2411.214 ms 2024-02-21T23:44:52.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1515 0.25 MB/sec execute 190 sec latency 2239.916 ms 2024-02-21T23:44:53.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1518 0.25 MB/sec execute 191 sec latency 2869.294 ms 2024-02-21T23:44:54.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1521 0.25 MB/sec execute 192 sec latency 3262.397 ms 2024-02-21T23:44:55.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1525 0.25 MB/sec execute 193 sec latency 3373.884 ms 2024-02-21T23:44:56.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1531 0.25 MB/sec execute 194 sec latency 3445.036 ms 2024-02-21T23:44:57.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1534 0.24 MB/sec execute 195 sec latency 2489.967 ms 2024-02-21T23:44:58.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1537 0.24 MB/sec execute 196 sec latency 2310.978 ms 2024-02-21T23:44:59.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1540 0.24 MB/sec execute 197 sec latency 3110.279 ms 2024-02-21T23:45:00.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1544 0.24 MB/sec execute 198 sec latency 3203.870 ms 2024-02-21T23:45:01.142 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:45:01.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:45:01.906 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1550 0.24 MB/sec execute 199 sec latency 2696.071 ms 2024-02-21T23:45:02.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1552 0.24 MB/sec execute 200 sec latency 2692.452 ms 2024-02-21T23:45:03.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1554 0.24 MB/sec execute 201 sec latency 3163.407 ms 2024-02-21T23:45:04.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1557 0.24 MB/sec execute 202 sec latency 3446.081 ms 2024-02-21T23:45:05.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1562 0.24 MB/sec execute 203 sec latency 2717.560 ms 2024-02-21T23:45:06.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1566 0.24 MB/sec execute 204 sec latency 2460.354 ms 2024-02-21T23:45:07.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1570 0.23 MB/sec execute 205 sec latency 3215.889 ms 2024-02-21T23:45:08.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1573 0.23 MB/sec execute 206 sec latency 3865.135 ms 2024-02-21T23:45:09.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1578 0.23 MB/sec execute 207 sec latency 3306.339 ms 2024-02-21T23:45:10.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1582 0.23 MB/sec execute 208 sec latency 2779.736 ms 2024-02-21T23:45:11.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1584 0.23 MB/sec execute 209 sec latency 2026.845 ms 2024-02-21T23:45:12.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1588 0.23 MB/sec execute 210 sec latency 2842.429 ms 2024-02-21T23:45:13.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1591 0.23 MB/sec execute 211 sec latency 3395.988 ms 2024-02-21T23:45:14.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1596 0.23 MB/sec execute 212 sec latency 3280.750 ms 2024-02-21T23:45:15.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1600 0.23 MB/sec execute 213 sec latency 3026.281 ms 2024-02-21T23:45:16.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1603 0.23 MB/sec execute 214 sec latency 2223.561 ms 2024-02-21T23:45:17.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1609 0.23 MB/sec execute 215 sec latency 2192.387 ms 2024-02-21T23:45:18.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1613 0.22 MB/sec execute 216 sec latency 2395.605 ms 2024-02-21T23:45:19.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1613 0.22 MB/sec execute 217 sec latency 2346.928 ms 2024-02-21T23:45:20.907 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1618 0.22 MB/sec execute 218 sec latency 2213.562 ms 2024-02-21T23:45:21.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1622 0.22 MB/sec execute 219 sec latency 1524.873 ms 2024-02-21T23:45:22.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1625 0.22 MB/sec execute 220 sec latency 1811.823 ms 2024-02-21T23:45:23.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1626 0.22 MB/sec execute 221 sec latency 2809.595 ms 2024-02-21T23:45:24.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1630 0.22 MB/sec execute 222 sec latency 2844.075 ms 2024-02-21T23:45:25.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1636 0.22 MB/sec execute 223 sec latency 2302.304 ms 2024-02-21T23:45:26.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1642 0.22 MB/sec execute 224 sec latency 1603.962 ms 2024-02-21T23:45:27.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1643 0.22 MB/sec execute 225 sec latency 2398.946 ms 2024-02-21T23:45:28.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1650 0.22 MB/sec execute 226 sec latency 2614.005 ms 2024-02-21T23:45:29.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1655 0.21 MB/sec execute 227 sec latency 2924.824 ms 2024-02-21T23:45:30.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1656 0.21 MB/sec execute 228 sec latency 1297.010 ms 2024-02-21T23:45:31.278 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:45:31.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:45:31.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1657 0.21 MB/sec execute 229 sec latency 2297.090 ms 2024-02-21T23:45:32.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1662 0.21 MB/sec execute 230 sec latency 2857.439 ms 2024-02-21T23:45:33.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1665 0.21 MB/sec execute 231 sec latency 3057.002 ms 2024-02-21T23:45:34.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1669 0.21 MB/sec execute 232 sec latency 2548.853 ms 2024-02-21T23:45:35.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1679 0.21 MB/sec execute 233 sec latency 3548.880 ms 2024-02-21T23:45:36.908 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1681 0.21 MB/sec execute 234 sec latency 3802.367 ms 2024-02-21T23:45:37.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1683 0.21 MB/sec execute 235 sec latency 3842.745 ms 2024-02-21T23:45:38.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1688 0.21 MB/sec execute 236 sec latency 3547.188 ms 2024-02-21T23:45:39.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1693 0.21 MB/sec execute 237 sec latency 3626.951 ms 2024-02-21T23:45:40.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1700 0.21 MB/sec execute 238 sec latency 2777.887 ms 2024-02-21T23:45:41.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1705 0.21 MB/sec execute 239 sec latency 1704.719 ms 2024-02-21T23:45:42.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1708 0.20 MB/sec execute 240 sec latency 2704.825 ms 2024-02-21T23:45:43.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1711 0.20 MB/sec execute 241 sec latency 3704.856 ms 2024-02-21T23:45:44.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1717 0.20 MB/sec execute 242 sec latency 4692.378 ms 2024-02-21T23:45:45.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1722 0.20 MB/sec execute 243 sec latency 4810.191 ms 2024-02-21T23:45:46.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1724 0.20 MB/sec execute 244 sec latency 3888.144 ms 2024-02-21T23:45:47.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1725 0.20 MB/sec execute 245 sec latency 4141.350 ms 2024-02-21T23:45:48.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1727 0.20 MB/sec execute 246 sec latency 4445.145 ms 2024-02-21T23:45:49.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1730 0.20 MB/sec execute 247 sec latency 3898.573 ms 2024-02-21T23:45:50.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1732 0.20 MB/sec execute 248 sec latency 3111.900 ms 2024-02-21T23:45:51.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1736 0.20 MB/sec execute 249 sec latency 3554.300 ms 2024-02-21T23:45:52.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1744 0.20 MB/sec execute 250 sec latency 3443.290 ms 2024-02-21T23:45:53.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1751 0.20 MB/sec execute 251 sec latency 3486.725 ms 2024-02-21T23:45:54.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1756 0.20 MB/sec execute 252 sec latency 2658.340 ms 2024-02-21T23:45:55.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1759 0.20 MB/sec execute 253 sec latency 3058.189 ms 2024-02-21T23:45:56.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1763 0.20 MB/sec execute 254 sec latency 2846.669 ms 2024-02-21T23:45:57.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1770 0.19 MB/sec execute 255 sec latency 2779.050 ms 2024-02-21T23:45:58.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1780 0.19 MB/sec execute 256 sec latency 2804.151 ms 2024-02-21T23:45:59.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1785 0.19 MB/sec execute 257 sec latency 3557.855 ms 2024-02-21T23:46:00.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1791 0.19 MB/sec execute 258 sec latency 4178.736 ms 2024-02-21T23:46:01.441 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:46:01.444 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:46:01.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1795 0.19 MB/sec execute 259 sec latency 4199.163 ms 2024-02-21T23:46:02.909 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1797 0.19 MB/sec execute 260 sec latency 4436.022 ms 2024-02-21T23:46:03.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1799 0.19 MB/sec execute 261 sec latency 3367.334 ms 2024-02-21T23:46:04.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1801 0.19 MB/sec execute 262 sec latency 3652.617 ms 2024-02-21T23:46:05.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1804 0.19 MB/sec execute 263 sec latency 3578.976 ms 2024-02-21T23:46:06.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1807 0.19 MB/sec execute 264 sec latency 3909.069 ms 2024-02-21T23:46:07.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1812 0.19 MB/sec execute 265 sec latency 4382.850 ms 2024-02-21T23:46:08.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1817 0.19 MB/sec execute 266 sec latency 2659.835 ms 2024-02-21T23:46:09.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1821 0.19 MB/sec execute 267 sec latency 2969.831 ms 2024-02-21T23:46:10.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1822 0.19 MB/sec execute 268 sec latency 2344.883 ms 2024-02-21T23:46:11.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1825 0.19 MB/sec execute 269 sec latency 2214.949 ms 2024-02-21T23:46:12.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1829 0.19 MB/sec execute 270 sec latency 2215.793 ms 2024-02-21T23:46:13.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1832 0.19 MB/sec execute 271 sec latency 2317.494 ms 2024-02-21T23:46:14.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1836 0.18 MB/sec execute 272 sec latency 2623.554 ms 2024-02-21T23:46:15.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1838 0.18 MB/sec execute 273 sec latency 2562.969 ms 2024-02-21T23:46:16.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1841 0.18 MB/sec execute 274 sec latency 2121.213 ms 2024-02-21T23:46:17.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1842 0.18 MB/sec execute 275 sec latency 2099.317 ms 2024-02-21T23:46:18.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1846 0.18 MB/sec execute 276 sec latency 2300.457 ms 2024-02-21T23:46:19.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1852 0.18 MB/sec execute 277 sec latency 2360.344 ms 2024-02-21T23:46:20.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1856 0.18 MB/sec execute 278 sec latency 1791.614 ms 2024-02-21T23:46:21.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1858 0.18 MB/sec execute 279 sec latency 1778.863 ms 2024-02-21T23:46:22.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1861 0.18 MB/sec execute 280 sec latency 2190.884 ms 2024-02-21T23:46:23.910 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1865 0.18 MB/sec execute 281 sec latency 2364.578 ms 2024-02-21T23:46:24.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1871 0.18 MB/sec execute 282 sec latency 2410.863 ms 2024-02-21T23:46:25.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1874 0.18 MB/sec execute 283 sec latency 2126.037 ms 2024-02-21T23:46:26.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1875 0.18 MB/sec execute 284 sec latency 1883.094 ms 2024-02-21T23:46:27.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1877 0.18 MB/sec execute 285 sec latency 2135.215 ms 2024-02-21T23:46:28.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1881 0.18 MB/sec execute 286 sec latency 2086.617 ms 2024-02-21T23:46:29.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1886 0.18 MB/sec execute 287 sec latency 2274.902 ms 2024-02-21T23:46:30.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1891 0.18 MB/sec execute 288 sec latency 2094.329 ms 2024-02-21T23:46:31.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:46:31.527 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:46:31.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1898 0.18 MB/sec execute 289 sec latency 1344.557 ms 2024-02-21T23:46:32.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1901 0.18 MB/sec execute 290 sec latency 2228.064 ms 2024-02-21T23:46:33.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1905 0.17 MB/sec execute 291 sec latency 2568.949 ms 2024-02-21T23:46:34.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1912 0.17 MB/sec execute 292 sec latency 2810.865 ms 2024-02-21T23:46:35.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1918 0.17 MB/sec execute 293 sec latency 2633.851 ms 2024-02-21T23:46:36.911 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1926 0.17 MB/sec execute 294 sec latency 2671.271 ms 2024-02-21T23:46:37.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1928 0.17 MB/sec execute 295 sec latency 3086.335 ms 2024-02-21T23:46:38.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1930 0.17 MB/sec execute 296 sec latency 3288.659 ms 2024-02-21T23:46:39.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1932 0.17 MB/sec execute 297 sec latency 3183.391 ms 2024-02-21T23:46:40.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1934 0.17 MB/sec execute 298 sec latency 2832.654 ms 2024-02-21T23:46:41.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1941 0.17 MB/sec execute 299 sec latency 2402.823 ms 2024-02-21T23:46:42.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1944 0.17 MB/sec execute 300 sec latency 3077.021 ms 2024-02-21T23:46:43.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1947 0.17 MB/sec execute 301 sec latency 3462.135 ms 2024-02-21T23:46:44.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1950 0.17 MB/sec execute 302 sec latency 3604.424 ms 2024-02-21T23:46:45.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1961 0.17 MB/sec execute 303 sec latency 2189.816 ms 2024-02-21T23:46:46.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1969 0.17 MB/sec execute 304 sec latency 2051.000 ms 2024-02-21T23:46:47.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1975 0.17 MB/sec execute 305 sec latency 2378.952 ms 2024-02-21T23:46:48.912 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1980 0.17 MB/sec execute 306 sec latency 3379.091 ms 2024-02-21T23:46:49.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1982 0.17 MB/sec execute 307 sec latency 3644.224 ms 2024-02-21T23:46:50.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1991 0.17 MB/sec execute 308 sec latency 3458.644 ms 2024-02-21T23:46:51.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1994 0.17 MB/sec execute 309 sec latency 2721.549 ms 2024-02-21T23:46:52.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1996 0.17 MB/sec execute 310 sec latency 3065.186 ms 2024-02-21T23:46:53.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1998 0.17 MB/sec execute 311 sec latency 4065.216 ms 2024-02-21T23:46:54.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2001 0.17 MB/sec execute 312 sec latency 4672.341 ms 2024-02-21T23:46:55.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2005 0.17 MB/sec execute 313 sec latency 4689.655 ms 2024-02-21T23:46:56.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2007 0.17 MB/sec execute 314 sec latency 4360.152 ms 2024-02-21T23:46:57.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2009 0.17 MB/sec execute 315 sec latency 5247.288 ms 2024-02-21T23:46:58.913 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2012 0.16 MB/sec execute 316 sec latency 3417.663 ms 2024-02-21T23:46:59.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2016 0.16 MB/sec execute 317 sec latency 2929.555 ms 2024-02-21T23:47:00.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2021 0.16 MB/sec execute 318 sec latency 2590.829 ms 2024-02-21T23:47:01.653 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:47:01.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:47:01.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2026 0.16 MB/sec execute 319 sec latency 2901.568 ms 2024-02-21T23:47:02.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2029 0.16 MB/sec execute 320 sec latency 2895.792 ms 2024-02-21T23:47:03.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2031 0.16 MB/sec execute 321 sec latency 3092.562 ms 2024-02-21T23:47:04.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2035 0.16 MB/sec execute 322 sec latency 3370.258 ms 2024-02-21T23:47:05.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2038 0.16 MB/sec execute 323 sec latency 2976.807 ms 2024-02-21T23:47:06.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2040 0.16 MB/sec execute 324 sec latency 2480.052 ms 2024-02-21T23:47:07.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2045 0.16 MB/sec execute 325 sec latency 2932.519 ms 2024-02-21T23:47:08.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2049 0.16 MB/sec execute 326 sec latency 1051.053 ms 2024-02-21T23:47:09.914 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2049 0.16 MB/sec execute 327 sec latency 2051.110 ms 2024-02-21T23:47:10.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2055 0.16 MB/sec execute 328 sec latency 2348.874 ms 2024-02-21T23:47:11.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2057 0.16 MB/sec execute 329 sec latency 2247.530 ms 2024-02-21T23:47:12.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2059 0.16 MB/sec execute 330 sec latency 2263.847 ms 2024-02-21T23:47:13.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2062 0.16 MB/sec execute 331 sec latency 2432.503 ms 2024-02-21T23:47:14.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2063 0.16 MB/sec execute 332 sec latency 2641.490 ms 2024-02-21T23:47:15.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2067 0.16 MB/sec execute 333 sec latency 2516.615 ms 2024-02-21T23:47:16.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2068 0.16 MB/sec execute 334 sec latency 2436.034 ms 2024-02-21T23:47:17.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2070 0.16 MB/sec execute 335 sec latency 2412.538 ms 2024-02-21T23:47:18.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2071 0.16 MB/sec execute 336 sec latency 2524.740 ms 2024-02-21T23:47:19.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2075 0.16 MB/sec execute 337 sec latency 2695.887 ms 2024-02-21T23:47:20.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2079 0.16 MB/sec execute 338 sec latency 2433.906 ms 2024-02-21T23:47:21.915 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2087 0.16 MB/sec execute 339 sec latency 2549.586 ms 2024-02-21T23:47:22.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2090 0.16 MB/sec execute 340 sec latency 1807.669 ms 2024-02-21T23:47:23.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2094 0.16 MB/sec execute 341 sec latency 2344.984 ms 2024-02-21T23:47:24.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2100 0.16 MB/sec execute 342 sec latency 2420.899 ms 2024-02-21T23:47:25.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2102 0.16 MB/sec execute 343 sec latency 2322.110 ms 2024-02-21T23:47:26.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2103 0.15 MB/sec execute 344 sec latency 2574.294 ms 2024-02-21T23:47:27.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2109 0.15 MB/sec execute 345 sec latency 3136.329 ms 2024-02-21T23:47:28.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2111 0.15 MB/sec execute 346 sec latency 3009.324 ms 2024-02-21T23:47:29.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2113 0.15 MB/sec execute 347 sec latency 4009.415 ms 2024-02-21T23:47:30.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2116 0.15 MB/sec execute 348 sec latency 4301.962 ms 2024-02-21T23:47:31.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:47:31.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:47:31.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2119 0.15 MB/sec execute 349 sec latency 3559.306 ms 2024-02-21T23:47:32.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2122 0.15 MB/sec execute 350 sec latency 2849.403 ms 2024-02-21T23:47:33.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2126 0.15 MB/sec execute 351 sec latency 2008.444 ms 2024-02-21T23:47:34.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2127 0.15 MB/sec execute 352 sec latency 3008.471 ms 2024-02-21T23:47:35.916 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2131 0.15 MB/sec execute 353 sec latency 3387.000 ms 2024-02-21T23:47:36.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2139 0.15 MB/sec execute 354 sec latency 2789.060 ms 2024-02-21T23:47:37.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2146 0.15 MB/sec execute 355 sec latency 1941.602 ms 2024-02-21T23:47:38.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2153 0.15 MB/sec execute 356 sec latency 2062.796 ms 2024-02-21T23:47:39.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2155 0.15 MB/sec execute 357 sec latency 2590.860 ms 2024-02-21T23:47:40.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2157 0.15 MB/sec execute 358 sec latency 2835.364 ms 2024-02-21T23:47:41.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2161 0.15 MB/sec execute 359 sec latency 2675.279 ms 2024-02-21T23:47:42.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2164 0.15 MB/sec execute 360 sec latency 2367.582 ms 2024-02-21T23:47:43.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2165 0.15 MB/sec execute 361 sec latency 3103.337 ms 2024-02-21T23:47:44.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2169 0.15 MB/sec execute 362 sec latency 3544.480 ms 2024-02-21T23:47:45.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2169 0.15 MB/sec execute 363 sec latency 4544.507 ms 2024-02-21T23:47:46.917 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2171 0.15 MB/sec execute 364 sec latency 4926.562 ms 2024-02-21T23:47:47.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2178 0.15 MB/sec execute 365 sec latency 4921.088 ms 2024-02-21T23:47:48.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2184 0.15 MB/sec execute 366 sec latency 4036.257 ms 2024-02-21T23:47:49.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2188 0.15 MB/sec execute 367 sec latency 2295.415 ms 2024-02-21T23:47:50.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2196 0.15 MB/sec execute 368 sec latency 2997.402 ms 2024-02-21T23:47:51.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2201 0.15 MB/sec execute 369 sec latency 2973.911 ms 2024-02-21T23:47:52.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2206 0.15 MB/sec execute 370 sec latency 2816.895 ms 2024-02-21T23:47:53.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2211 0.15 MB/sec execute 371 sec latency 2347.447 ms 2024-02-21T23:47:54.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2213 0.15 MB/sec execute 372 sec latency 2489.782 ms 2024-02-21T23:47:55.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2219 0.15 MB/sec execute 373 sec latency 2616.763 ms 2024-02-21T23:47:56.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2224 0.15 MB/sec execute 374 sec latency 2652.945 ms 2024-02-21T23:47:57.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2226 0.15 MB/sec execute 375 sec latency 2237.579 ms 2024-02-21T23:47:58.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2232 0.16 MB/sec execute 376 sec latency 2598.851 ms 2024-02-21T23:47:59.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2235 0.15 MB/sec execute 377 sec latency 3104.561 ms 2024-02-21T23:48:00.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2239 0.15 MB/sec execute 378 sec latency 3327.522 ms 2024-02-21T23:48:01.857 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:48:01.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:48:01.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2243 0.15 MB/sec execute 379 sec latency 2692.298 ms 2024-02-21T23:48:02.918 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2249 0.15 MB/sec execute 380 sec latency 2631.162 ms 2024-02-21T23:48:03.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2258 0.15 MB/sec execute 381 sec latency 2856.891 ms 2024-02-21T23:48:04.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2263 0.16 MB/sec execute 382 sec latency 2594.952 ms 2024-02-21T23:48:05.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2269 0.16 MB/sec execute 383 sec latency 2684.668 ms 2024-02-21T23:48:06.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2274 0.16 MB/sec execute 384 sec latency 2887.942 ms 2024-02-21T23:48:07.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2285 0.16 MB/sec execute 385 sec latency 2719.886 ms 2024-02-21T23:48:08.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2289 0.16 MB/sec execute 386 sec latency 2817.133 ms 2024-02-21T23:48:09.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2298 0.16 MB/sec execute 387 sec latency 3166.334 ms 2024-02-21T23:48:10.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2305 0.17 MB/sec execute 388 sec latency 3340.486 ms 2024-02-21T23:48:11.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2306 0.17 MB/sec execute 389 sec latency 3685.564 ms 2024-02-21T23:48:12.919 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2307 0.17 MB/sec execute 390 sec latency 2924.222 ms 2024-02-21T23:48:13.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2308 0.17 MB/sec execute 391 sec latency 3680.052 ms 2024-02-21T23:48:14.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2310 0.16 MB/sec execute 392 sec latency 4056.081 ms 2024-02-21T23:48:15.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2312 0.16 MB/sec execute 393 sec latency 4073.869 ms 2024-02-21T23:48:16.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2314 0.16 MB/sec execute 394 sec latency 4231.358 ms 2024-02-21T23:48:17.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2316 0.16 MB/sec execute 395 sec latency 3526.742 ms 2024-02-21T23:48:18.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2318 0.16 MB/sec execute 396 sec latency 2446.210 ms 2024-02-21T23:48:19.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2319 0.16 MB/sec execute 397 sec latency 2462.525 ms 2024-02-21T23:48:20.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2324 0.16 MB/sec execute 398 sec latency 2403.959 ms 2024-02-21T23:48:21.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2326 0.16 MB/sec execute 399 sec latency 2596.445 ms 2024-02-21T23:48:22.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2327 0.16 MB/sec execute 400 sec latency 2245.758 ms 2024-02-21T23:48:23.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2335 0.16 MB/sec execute 401 sec latency 2327.120 ms 2024-02-21T23:48:24.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2335 0.16 MB/sec execute 402 sec latency 2658.592 ms 2024-02-21T23:48:25.920 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2336 0.16 MB/sec execute 403 sec latency 2852.254 ms 2024-02-21T23:48:26.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2338 0.16 MB/sec execute 404 sec latency 2862.166 ms 2024-02-21T23:48:27.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2341 0.16 MB/sec execute 405 sec latency 2759.176 ms 2024-02-21T23:48:28.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2343 0.16 MB/sec execute 406 sec latency 2341.741 ms 2024-02-21T23:48:29.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2346 0.16 MB/sec execute 407 sec latency 2451.185 ms 2024-02-21T23:48:30.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2347 0.16 MB/sec execute 408 sec latency 2543.424 ms 2024-02-21T23:48:31.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:48:31.916 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:48:31.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2348 0.16 MB/sec execute 409 sec latency 3029.252 ms 2024-02-21T23:48:32.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2350 0.16 MB/sec execute 410 sec latency 2783.022 ms 2024-02-21T23:48:33.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2354 0.16 MB/sec execute 411 sec latency 2662.088 ms 2024-02-21T23:48:34.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2356 0.16 MB/sec execute 412 sec latency 2445.534 ms 2024-02-21T23:48:35.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2357 0.16 MB/sec execute 413 sec latency 2815.209 ms 2024-02-21T23:48:36.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2360 0.16 MB/sec execute 414 sec latency 3096.865 ms 2024-02-21T23:48:37.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2365 0.16 MB/sec execute 415 sec latency 3123.852 ms 2024-02-21T23:48:38.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2369 0.16 MB/sec execute 416 sec latency 2213.848 ms 2024-02-21T23:48:39.921 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2371 0.16 MB/sec execute 417 sec latency 2456.624 ms 2024-02-21T23:48:40.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2373 0.16 MB/sec execute 418 sec latency 2747.592 ms 2024-02-21T23:48:41.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2374 0.16 MB/sec execute 419 sec latency 2839.942 ms 2024-02-21T23:48:42.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2377 0.16 MB/sec execute 420 sec latency 2655.104 ms 2024-02-21T23:48:43.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2379 0.16 MB/sec execute 421 sec latency 2521.263 ms 2024-02-21T23:48:44.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2381 0.16 MB/sec execute 422 sec latency 2875.817 ms 2024-02-21T23:48:45.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2382 0.16 MB/sec execute 423 sec latency 2970.191 ms 2024-02-21T23:48:46.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2385 0.16 MB/sec execute 424 sec latency 2887.174 ms 2024-02-21T23:48:47.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2387 0.16 MB/sec execute 425 sec latency 2888.441 ms 2024-02-21T23:48:48.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2389 0.16 MB/sec execute 426 sec latency 2291.585 ms 2024-02-21T23:48:49.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2392 0.16 MB/sec execute 427 sec latency 2568.650 ms 2024-02-21T23:48:50.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2396 0.16 MB/sec execute 428 sec latency 2827.034 ms 2024-02-21T23:48:51.922 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2397 0.16 MB/sec execute 429 sec latency 2391.820 ms 2024-02-21T23:48:52.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2398 0.16 MB/sec execute 430 sec latency 2473.617 ms 2024-02-21T23:48:53.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2405 0.16 MB/sec execute 431 sec latency 2625.435 ms 2024-02-21T23:48:54.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2409 0.16 MB/sec execute 432 sec latency 1701.601 ms 2024-02-21T23:48:55.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2410 0.15 MB/sec execute 433 sec latency 2701.637 ms 2024-02-21T23:48:56.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2413 0.15 MB/sec execute 434 sec latency 3474.093 ms 2024-02-21T23:48:57.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2416 0.16 MB/sec execute 435 sec latency 1699.656 ms 2024-02-21T23:48:58.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2418 0.16 MB/sec execute 436 sec latency 2691.076 ms 2024-02-21T23:48:59.195 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:24:27.099 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:48:59.199 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:24:27.104 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:48:59.224 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:24:27.133 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:48:59.224 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:24:27.133 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:48:59.228 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:24:27.137 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:48:59.250 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:24:27.159 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:48:59.328 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:24:27.232 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:48:59.352 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:24:27.261 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:48:59.366 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:24:27.275 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:48:59.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2421 0.15 MB/sec execute 437 sec latency 2712.731 ms 2024-02-21T23:49:00.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2424 0.16 MB/sec execute 438 sec latency 2565.824 ms 2024-02-21T23:49:01.923 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2428 0.16 MB/sec execute 439 sec latency 3108.916 ms 2024-02-21T23:49:02.088 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:49:02.091 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:49:02.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2432 0.16 MB/sec execute 440 sec latency 3381.103 ms 2024-02-21T23:49:03.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2435 0.16 MB/sec execute 441 sec latency 2317.371 ms 2024-02-21T23:49:04.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2438 0.16 MB/sec execute 442 sec latency 2609.099 ms 2024-02-21T23:49:05.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2439 0.16 MB/sec execute 443 sec latency 2185.443 ms 2024-02-21T23:49:06.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2441 0.16 MB/sec execute 444 sec latency 2739.361 ms 2024-02-21T23:49:07.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2449 0.16 MB/sec execute 445 sec latency 3127.909 ms 2024-02-21T23:49:08.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2451 0.16 MB/sec execute 446 sec latency 2606.424 ms 2024-02-21T23:49:09.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2454 0.16 MB/sec execute 447 sec latency 3432.940 ms 2024-02-21T23:49:10.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2456 0.16 MB/sec execute 448 sec latency 3327.487 ms 2024-02-21T23:49:11.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2457 0.16 MB/sec execute 449 sec latency 2822.779 ms 2024-02-21T23:49:12.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2459 0.16 MB/sec execute 450 sec latency 2391.333 ms 2024-02-21T23:49:13.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2463 0.16 MB/sec execute 451 sec latency 2251.896 ms 2024-02-21T23:49:14.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2470 0.16 MB/sec execute 452 sec latency 2663.315 ms 2024-02-21T23:49:15.924 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2472 0.16 MB/sec execute 453 sec latency 1646.652 ms 2024-02-21T23:49:16.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2475 0.16 MB/sec execute 454 sec latency 2646.678 ms 2024-02-21T23:49:17.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2484 0.16 MB/sec execute 455 sec latency 3006.916 ms 2024-02-21T23:49:18.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2486 0.16 MB/sec execute 456 sec latency 1561.680 ms 2024-02-21T23:49:19.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2487 0.16 MB/sec execute 457 sec latency 2561.740 ms 2024-02-21T23:49:20.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2498 0.16 MB/sec execute 458 sec latency 3027.574 ms 2024-02-21T23:49:21.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2501 0.16 MB/sec execute 459 sec latency 2665.380 ms 2024-02-21T23:49:22.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2503 0.16 MB/sec execute 460 sec latency 2930.775 ms 2024-02-21T23:49:23.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2506 0.16 MB/sec execute 461 sec latency 2628.503 ms 2024-02-21T23:49:24.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2508 0.16 MB/sec execute 462 sec latency 2922.958 ms 2024-02-21T23:49:25.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2513 0.16 MB/sec execute 463 sec latency 3130.513 ms 2024-02-21T23:49:26.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2519 0.16 MB/sec execute 464 sec latency 2950.062 ms 2024-02-21T23:49:27.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2522 0.16 MB/sec execute 465 sec latency 3031.248 ms 2024-02-21T23:49:28.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2524 0.16 MB/sec execute 466 sec latency 2806.900 ms 2024-02-21T23:49:29.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2525 0.16 MB/sec execute 467 sec latency 2356.717 ms 2024-02-21T23:49:30.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2532 0.16 MB/sec execute 468 sec latency 2421.962 ms 2024-02-21T23:49:31.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2535 0.16 MB/sec execute 469 sec latency 2258.254 ms 2024-02-21T23:49:32.266 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:49:32.268 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:49:32.925 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2541 0.16 MB/sec execute 470 sec latency 1887.266 ms 2024-02-21T23:49:33.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2543 0.16 MB/sec execute 471 sec latency 2099.331 ms 2024-02-21T23:49:34.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2545 0.16 MB/sec execute 472 sec latency 2462.315 ms 2024-02-21T23:49:35.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2555 0.16 MB/sec execute 473 sec latency 3105.855 ms 2024-02-21T23:49:36.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2562 0.16 MB/sec execute 474 sec latency 2842.493 ms 2024-02-21T23:49:37.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2568 0.16 MB/sec execute 475 sec latency 2991.524 ms 2024-02-21T23:49:38.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2570 0.16 MB/sec execute 476 sec latency 2669.501 ms 2024-02-21T23:49:39.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2573 0.16 MB/sec execute 477 sec latency 2625.712 ms 2024-02-21T23:49:40.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2575 0.16 MB/sec execute 478 sec latency 2815.383 ms 2024-02-21T23:49:41.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2576 0.16 MB/sec execute 479 sec latency 2682.232 ms 2024-02-21T23:49:42.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2579 0.16 MB/sec execute 480 sec latency 2714.589 ms 2024-02-21T23:49:43.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2582 0.16 MB/sec execute 481 sec latency 2764.669 ms 2024-02-21T23:49:44.926 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2583 0.16 MB/sec execute 482 sec latency 2737.697 ms 2024-02-21T23:49:45.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2585 0.16 MB/sec execute 483 sec latency 3394.559 ms 2024-02-21T23:49:46.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2587 0.16 MB/sec execute 484 sec latency 3146.070 ms 2024-02-21T23:49:47.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2591 0.16 MB/sec execute 485 sec latency 2552.242 ms 2024-02-21T23:49:48.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2594 0.16 MB/sec execute 486 sec latency 2814.820 ms 2024-02-21T23:49:49.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2596 0.16 MB/sec execute 487 sec latency 3045.252 ms 2024-02-21T23:49:50.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2598 0.16 MB/sec execute 488 sec latency 2720.426 ms 2024-02-21T23:49:51.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2603 0.16 MB/sec execute 489 sec latency 2871.838 ms 2024-02-21T23:49:52.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2610 0.16 MB/sec execute 490 sec latency 2639.367 ms 2024-02-21T23:49:53.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2612 0.16 MB/sec execute 491 sec latency 2674.124 ms 2024-02-21T23:49:54.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2624 0.17 MB/sec execute 492 sec latency 2474.245 ms 2024-02-21T23:49:55.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2629 0.17 MB/sec execute 493 sec latency 2731.194 ms 2024-02-21T23:49:56.927 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2631 0.17 MB/sec execute 494 sec latency 3154.974 ms 2024-02-21T23:49:57.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2643 0.17 MB/sec execute 495 sec latency 3274.595 ms 2024-02-21T23:49:58.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2647 0.17 MB/sec execute 496 sec latency 3297.017 ms 2024-02-21T23:49:59.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2649 0.17 MB/sec execute 497 sec latency 2292.383 ms 2024-02-21T23:50:00.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2653 0.17 MB/sec execute 498 sec latency 2478.004 ms 2024-02-21T23:50:01.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2657 0.17 MB/sec execute 499 sec latency 2597.711 ms 2024-02-21T23:50:02.325 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:50:02.328 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:50:02.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2668 0.18 MB/sec execute 500 sec latency 2839.949 ms 2024-02-21T23:50:03.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2669 0.18 MB/sec execute 501 sec latency 2371.992 ms 2024-02-21T23:50:04.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2675 0.18 MB/sec execute 502 sec latency 2507.831 ms 2024-02-21T23:50:05.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2685 0.18 MB/sec execute 503 sec latency 2569.414 ms 2024-02-21T23:50:06.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2686 0.18 MB/sec execute 504 sec latency 2665.238 ms 2024-02-21T23:50:07.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2691 0.18 MB/sec execute 505 sec latency 2636.403 ms 2024-02-21T23:50:08.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2701 0.19 MB/sec execute 506 sec latency 2446.379 ms 2024-02-21T23:50:09.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2713 0.19 MB/sec execute 507 sec latency 2447.068 ms 2024-02-21T23:50:10.928 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2714 0.19 MB/sec execute 508 sec latency 2694.888 ms 2024-02-21T23:50:11.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2720 0.19 MB/sec execute 509 sec latency 2427.498 ms 2024-02-21T23:50:12.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2758 0.20 MB/sec execute 510 sec latency 2476.069 ms 2024-02-21T23:50:13.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2768 0.21 MB/sec execute 511 sec latency 2576.609 ms 2024-02-21T23:50:14.432 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:25:42.335 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:50:14.435 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:25:42.338 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:50:14.467 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:25:42.369 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:50:14.491 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:25:42.398 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:50:14.493 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:25:42.400 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:50:14.494 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:25:42.401 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:50:14.496 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:25:42.403 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:50:14.530 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:25:42.437 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:50:14.530 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:25:42.437 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:50:14.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2780 0.21 MB/sec execute 512 sec latency 2646.455 ms 2024-02-21T23:50:15.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2787 0.21 MB/sec execute 513 sec latency 2623.115 ms 2024-02-21T23:50:16.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2794 0.21 MB/sec execute 514 sec latency 2707.122 ms 2024-02-21T23:50:17.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2798 0.21 MB/sec execute 515 sec latency 2742.418 ms 2024-02-21T23:50:18.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2803 0.21 MB/sec execute 516 sec latency 2407.997 ms 2024-02-21T23:50:19.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2816 0.21 MB/sec execute 517 sec latency 917.862 ms 2024-02-21T23:50:20.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2818 0.21 MB/sec execute 518 sec latency 1917.892 ms 2024-02-21T23:50:21.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2822 0.21 MB/sec execute 519 sec latency 2917.924 ms 2024-02-21T23:50:22.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2825 0.21 MB/sec execute 520 sec latency 3391.028 ms 2024-02-21T23:50:23.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2836 0.21 MB/sec execute 521 sec latency 3013.419 ms 2024-02-21T23:50:24.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2839 0.21 MB/sec execute 522 sec latency 2763.977 ms 2024-02-21T23:50:25.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2846 0.21 MB/sec execute 523 sec latency 2586.565 ms 2024-02-21T23:50:26.929 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2857 0.21 MB/sec execute 524 sec latency 3334.533 ms 2024-02-21T23:50:27.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2860 0.21 MB/sec execute 525 sec latency 2809.524 ms 2024-02-21T23:50:28.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2865 0.21 MB/sec execute 526 sec latency 3271.992 ms 2024-02-21T23:50:29.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2869 0.22 MB/sec execute 527 sec latency 2756.450 ms 2024-02-21T23:50:30.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2873 0.22 MB/sec execute 528 sec latency 3006.826 ms 2024-02-21T23:50:31.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2894 0.22 MB/sec execute 529 sec latency 2990.855 ms 2024-02-21T23:50:32.689 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:50:32.692 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:50:32.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2908 0.22 MB/sec execute 530 sec latency 3115.981 ms 2024-02-21T23:50:33.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2917 0.22 MB/sec execute 531 sec latency 2607.672 ms 2024-02-21T23:50:34.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2924 0.22 MB/sec execute 532 sec latency 2525.141 ms 2024-02-21T23:50:35.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2927 0.22 MB/sec execute 533 sec latency 2844.674 ms 2024-02-21T23:50:36.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2931 0.22 MB/sec execute 534 sec latency 3179.402 ms 2024-02-21T23:50:37.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2934 0.22 MB/sec execute 535 sec latency 3220.479 ms 2024-02-21T23:50:38.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2938 0.22 MB/sec execute 536 sec latency 3027.622 ms 2024-02-21T23:50:39.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2945 0.22 MB/sec execute 537 sec latency 2747.397 ms 2024-02-21T23:50:40.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2950 0.23 MB/sec execute 538 sec latency 2636.393 ms 2024-02-21T23:50:41.930 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2955 0.22 MB/sec execute 539 sec latency 2977.717 ms 2024-02-21T23:50:42.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2962 0.23 MB/sec execute 540 sec latency 3196.041 ms 2024-02-21T23:50:43.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2964 0.23 MB/sec execute 541 sec latency 2917.616 ms 2024-02-21T23:50:44.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2967 0.22 MB/sec execute 542 sec latency 2950.076 ms 2024-02-21T23:50:45.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2970 0.22 MB/sec execute 543 sec latency 3285.114 ms 2024-02-21T23:50:46.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2972 0.22 MB/sec execute 544 sec latency 3541.155 ms 2024-02-21T23:50:47.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2977 0.22 MB/sec execute 545 sec latency 3699.856 ms 2024-02-21T23:50:48.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2984 0.22 MB/sec execute 546 sec latency 3308.278 ms 2024-02-21T23:50:49.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3003 0.22 MB/sec execute 547 sec latency 2472.516 ms 2024-02-21T23:50:50.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3006 0.22 MB/sec execute 548 sec latency 2185.463 ms 2024-02-21T23:50:51.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3023 0.22 MB/sec execute 549 sec latency 2152.820 ms 2024-02-21T23:50:52.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3025 0.22 MB/sec execute 550 sec latency 1901.024 ms 2024-02-21T23:50:53.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3027 0.22 MB/sec execute 551 sec latency 2901.115 ms 2024-02-21T23:50:54.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3030 0.22 MB/sec execute 552 sec latency 2948.212 ms 2024-02-21T23:50:55.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3031 0.22 MB/sec execute 553 sec latency 2623.123 ms 2024-02-21T23:50:56.931 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3050 0.22 MB/sec execute 554 sec latency 2835.620 ms 2024-02-21T23:50:57.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3053 0.22 MB/sec execute 555 sec latency 2512.520 ms 2024-02-21T23:50:58.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3072 0.22 MB/sec execute 556 sec latency 2445.116 ms 2024-02-21T23:50:59.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3074 0.22 MB/sec execute 557 sec latency 1925.787 ms 2024-02-21T23:51:00.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3077 0.22 MB/sec execute 558 sec latency 2809.198 ms 2024-02-21T23:51:01.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3082 0.22 MB/sec execute 559 sec latency 3083.114 ms 2024-02-21T23:51:02.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:51:02.752 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:51:02.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3089 0.23 MB/sec execute 560 sec latency 2947.080 ms 2024-02-21T23:51:03.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3092 0.22 MB/sec execute 561 sec latency 2658.040 ms 2024-02-21T23:51:04.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3111 0.23 MB/sec execute 562 sec latency 2889.826 ms 2024-02-21T23:51:05.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3116 0.23 MB/sec execute 563 sec latency 3223.304 ms 2024-02-21T23:51:06.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3134 0.23 MB/sec execute 564 sec latency 3349.031 ms 2024-02-21T23:51:07.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3136 0.23 MB/sec execute 565 sec latency 3413.086 ms 2024-02-21T23:51:08.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3140 0.23 MB/sec execute 566 sec latency 3065.366 ms 2024-02-21T23:51:09.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3147 0.23 MB/sec execute 567 sec latency 3164.195 ms 2024-02-21T23:51:10.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3149 0.23 MB/sec execute 568 sec latency 3040.128 ms 2024-02-21T23:51:11.932 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3166 0.23 MB/sec execute 569 sec latency 3195.556 ms 2024-02-21T23:51:12.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3199 0.23 MB/sec execute 570 sec latency 3243.205 ms 2024-02-21T23:51:13.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3214 0.23 MB/sec execute 571 sec latency 2786.428 ms 2024-02-21T23:51:14.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3216 0.23 MB/sec execute 572 sec latency 2344.739 ms 2024-02-21T23:51:15.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3218 0.23 MB/sec execute 573 sec latency 2885.695 ms 2024-02-21T23:51:16.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3223 0.23 MB/sec execute 574 sec latency 3044.478 ms 2024-02-21T23:51:17.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3228 0.23 MB/sec execute 575 sec latency 2665.378 ms 2024-02-21T23:51:18.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3235 0.23 MB/sec execute 576 sec latency 3003.170 ms 2024-02-21T23:51:19.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3241 0.24 MB/sec execute 577 sec latency 3190.480 ms 2024-02-21T23:51:20.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3243 0.24 MB/sec execute 578 sec latency 2549.269 ms 2024-02-21T23:51:21.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3247 0.23 MB/sec execute 579 sec latency 2529.018 ms 2024-02-21T23:51:22.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3252 0.23 MB/sec execute 580 sec latency 2513.026 ms 2024-02-21T23:51:23.933 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3256 0.24 MB/sec execute 581 sec latency 2724.762 ms 2024-02-21T23:51:24.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3264 0.24 MB/sec execute 582 sec latency 3003.058 ms 2024-02-21T23:51:25.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3273 0.24 MB/sec execute 583 sec latency 3238.051 ms 2024-02-21T23:51:26.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3276 0.24 MB/sec execute 584 sec latency 2903.477 ms 2024-02-21T23:51:27.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3276 0.24 MB/sec execute 585 sec latency 2896.638 ms 2024-02-21T23:51:28.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3277 0.24 MB/sec execute 586 sec latency 2207.996 ms 2024-02-21T23:51:29.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3278 0.24 MB/sec execute 587 sec latency 2318.913 ms 2024-02-21T23:51:30.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3280 0.24 MB/sec execute 588 sec latency 2413.343 ms 2024-02-21T23:51:31.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3282 0.24 MB/sec execute 589 sec latency 2750.045 ms 2024-02-21T23:51:32.883 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:51:32.886 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:51:32.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3283 0.24 MB/sec execute 590 sec latency 2520.718 ms 2024-02-21T23:51:33.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3284 0.24 MB/sec execute 591 sec latency 2589.631 ms 2024-02-21T23:51:34.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3285 0.24 MB/sec execute 592 sec latency 2382.933 ms 2024-02-21T23:51:35.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3285 0.24 MB/sec execute 593 sec latency 2226.938 ms 2024-02-21T23:51:36.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3286 0.24 MB/sec execute 594 sec latency 2425.758 ms 2024-02-21T23:51:37.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3286 0.24 MB/sec execute 595 sec latency 2325.671 ms 2024-02-21T23:51:38.934 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3287 0.24 MB/sec execute 596 sec latency 2328.575 ms 2024-02-21T23:51:39.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3287 0.23 MB/sec execute 597 sec latency 2219.889 ms 2024-02-21T23:51:40.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3287 0.23 MB/sec execute 598 sec latency 2260.093 ms 2024-02-21T23:51:41.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3288 0.23 MB/sec execute 599 sec latency 2356.326 ms 2024-02-21T23:51:42.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 600 sec 2024-02-21T23:51:43.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 601 sec 2024-02-21T23:51:44.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 602 sec 2024-02-21T23:51:45.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 603 sec 2024-02-21T23:51:46.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 604 sec 2024-02-21T23:51:47.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 605 sec 2024-02-21T23:51:48.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 606 sec 2024-02-21T23:51:49.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 607 sec 2024-02-21T23:51:50.935 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 608 sec 2024-02-21T23:51:51.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 609 sec 2024-02-21T23:51:52.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 610 sec 2024-02-21T23:51:53.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 611 sec 2024-02-21T23:51:54.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 612 sec 2024-02-21T23:51:55.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 613 sec 2024-02-21T23:51:56.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 614 sec 2024-02-21T23:51:57.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 615 sec 2024-02-21T23:51:58.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 616 sec 2024-02-21T23:51:59.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 617 sec 2024-02-21T23:52:00.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 618 sec 2024-02-21T23:52:01.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 619 sec 2024-02-21T23:52:02.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 620 sec 2024-02-21T23:52:02.965 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:52:02.968 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:52:03.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 621 sec 2024-02-21T23:52:04.936 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 622 sec 2024-02-21T23:52:05.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 623 sec 2024-02-21T23:52:06.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 624 sec 2024-02-21T23:52:07.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 625 sec 2024-02-21T23:52:08.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 626 sec 2024-02-21T23:52:09.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 627 sec 2024-02-21T23:52:10.940 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 628 sec 2024-02-21T23:52:11.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 629 sec 2024-02-21T23:52:12.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 630 sec 2024-02-21T23:52:13.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 631 sec 2024-02-21T23:52:14.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 632 sec 2024-02-21T23:52:15.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 633 sec 2024-02-21T23:52:16.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 634 sec 2024-02-21T23:52:17.937 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 635 sec 2024-02-21T23:52:18.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 636 sec 2024-02-21T23:52:19.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 637 sec 2024-02-21T23:52:20.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 638 sec 2024-02-21T23:52:21.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 639 sec 2024-02-21T23:52:22.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 640 sec 2024-02-21T23:52:23.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 641 sec 2024-02-21T23:52:24.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 642 sec 2024-02-21T23:52:25.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 643 sec 2024-02-21T23:52:26.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 644 sec 2024-02-21T23:52:27.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 645 sec 2024-02-21T23:52:28.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 646 sec 2024-02-21T23:52:29.938 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 647 sec 2024-02-21T23:52:30.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 648 sec 2024-02-21T23:52:31.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 649 sec 2024-02-21T23:52:32.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 650 sec 2024-02-21T23:52:33.246 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:52:33.249 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:52:33.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 651 sec 2024-02-21T23:52:34.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 652 sec 2024-02-21T23:52:35.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 653 sec 2024-02-21T23:52:36.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 654 sec 2024-02-21T23:52:37.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 655 sec 2024-02-21T23:52:38.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 656 sec 2024-02-21T23:52:39.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 657 sec 2024-02-21T23:52:40.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 658 sec 2024-02-21T23:52:41.340 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:28:09.240 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:52:41.345 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:28:09.245 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:52:41.351 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:28:09.255 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:52:41.354 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:28:09.258 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:52:41.355 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:28:09.259 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:52:41.358 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:28:09.262 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:52:41.406 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:28:09.306 37032== Warning: unimplemented fcntl command: 1036 2024-02-21T23:52:41.417 INFO:tasks.ceph.mon.c.smithi142.stderr:==00:00:28:09.321 36936== Warning: unimplemented fcntl command: 1036 2024-02-21T23:52:41.440 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:28:09.344 36935== Warning: unimplemented fcntl command: 1036 2024-02-21T23:52:41.939 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 659 sec 2024-02-21T23:52:42.940 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 660 sec 2024-02-21T23:52:43.940 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 661 sec 2024-02-21T23:52:44.940 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 662 sec 2024-02-21T23:52:45.940 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 663 sec 2024-02-21T23:52:46.940 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 664 sec 2024-02-21T23:52:47.940 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 665 sec 2024-02-21T23:52:48.940 INFO:tasks.workunit.client.0.smithi133.stdout: 8 cleanup 666 sec 2024-02-21T23:52:49.940 INFO:tasks.workunit.client.0.smithi133.stdout: 7 cleanup 667 sec 2024-02-21T23:52:50.940 INFO:tasks.workunit.client.0.smithi133.stdout: 7 cleanup 668 sec 2024-02-21T23:52:51.940 INFO:tasks.workunit.client.0.smithi133.stdout: 7 cleanup 669 sec 2024-02-21T23:52:52.940 INFO:tasks.workunit.client.0.smithi133.stdout: 3 cleanup 670 sec 2024-02-21T23:52:53.695 INFO:tasks.workunit.client.0.smithi133.stdout: 0 cleanup 671 sec 2024-02-21T23:52:53.695 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-02-21T23:52:53.695 INFO:tasks.workunit.client.0.smithi133.stdout: Operation Count AvgLat MaxLat 2024-02-21T23:52:53.695 INFO:tasks.workunit.client.0.smithi133.stdout: ---------------------------------------- 2024-02-21T23:52:53.695 INFO:tasks.workunit.client.0.smithi133.stdout: NTCreateX 3922 295.082 2672.308 2024-02-21T23:52:53.695 INFO:tasks.workunit.client.0.smithi133.stdout: Close 3224 3.134 84.084 2024-02-21T23:52:53.695 INFO:tasks.workunit.client.0.smithi133.stdout: Rename 209 856.471 1878.666 2024-02-21T23:52:53.695 INFO:tasks.workunit.client.0.smithi133.stdout: Unlink 466 282.972 1286.703 2024-02-21T23:52:53.695 INFO:tasks.workunit.client.0.smithi133.stdout: Qpathinfo 4014 190.831 2283.919 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: Qfileinfo 851 6.196 86.783 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: Qfsinfo 566 22.502 192.434 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: Sfileinfo 452 40.193 151.354 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: Find 1370 2642.079 5247.281 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: WriteX 2381 13.201 379.864 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: ReadX 7746 0.069 25.735 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: LockX 20 133.865 477.972 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: UnlockX 20 112.736 246.891 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: Flush 293 258.487 661.319 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-02-21T23:52:53.696 INFO:tasks.workunit.client.0.smithi133.stdout:Throughput 0.234068 MB/sec 10 clients 10 procs max_latency=5247.288 ms 2024-02-21T23:52:53.697 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-21T23:52:53.697 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-21T23:53:03.360 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:53:03.363 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:53:16.291 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-02-21T23:53:16.291 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-21T23:53:16.553 DEBUG:teuthology.parallel:result is None 2024-02-21T23:53:16.553 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-21T23:53:16.736 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-21T23:53:16.736 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-02-21T23:53:16.747 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-02-21T23:53:16.748 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:53:16.748 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-21T23:53:16.762 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:53:16.762 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-21T23:53:16.818 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:53:16.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.966+0000 7fc3d1955640 1 Processor -- start 2024-02-21T23:53:16.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.966+0000 7fc3d1955640 1 -- start start 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.966+0000 7fc3d1955640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc3cc154600 0x7fc3cc0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.966+0000 7fc3d1955640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc3cc0c6f50 0x7fc3cc0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.966+0000 7fc3d1955640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 0x7fc3cc0c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.966+0000 7fc3d1955640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3cc0c74a0 con 0x7fc3cc154600 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.966+0000 7fc3d1955640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3cc0c7780 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.966+0000 7fc3d1955640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc15ab40 con 0x7fc3cc0cab80 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.967+0000 7fc3ca7fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc3cc0c6f50 0x7fc3cc0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.967+0000 7fc3caffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc3cc154600 0x7fc3cc0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.967+0000 7fc3cb7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 0x7fc3cc0c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.967+0000 7fc3ca7fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc3cc0c6f50 0x7fc3cc0ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:52402/0 (socket says 172.21.15.133:52402) 2024-02-21T23:53:16.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.967+0000 7fc3ca7fc640 1 -- 172.21.15.133:0/2781166876 learned_addr learned my addr 172.21.15.133:0/2781166876 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:16.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.970+0000 7fc3ca7fc640 1 -- 172.21.15.133:0/2781166876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 msgr2=0x7fc3cc0c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:16.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.970+0000 7fc3ca7fc640 1 --2- 172.21.15.133:0/2781166876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 0x7fc3cc0c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:16.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.970+0000 7fc3ca7fc640 1 -- 172.21.15.133:0/2781166876 >> v2:172.21.15.133:3300/0 conn(0x7fc3cc154600 msgr2=0x7fc3cc0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:16.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.970+0000 7fc3ca7fc640 1 --2- 172.21.15.133:0/2781166876 >> v2:172.21.15.133:3300/0 conn(0x7fc3cc154600 0x7fc3cc0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:16.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.970+0000 7fc3ca7fc640 1 -- 172.21.15.133:0/2781166876 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc15ae00 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.972+0000 7fc3ca7fc640 1 --2- 172.21.15.133:0/2781166876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 0x7fc3cc0ca640 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc06d390 tx=0x7fc3c002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7792ee41c06667fa server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:16.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.975+0000 7fc3c9ffb640 1 -- 172.21.15.133:0/2781166876 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0035020 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.975+0000 7fc3c9ffb640 1 -- 172.21.15.133:0/2781166876 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c0031870 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.975+0000 7fc3c9ffb640 1 -- 172.21.15.133:0/2781166876 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0031c40 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.976+0000 7fc3d1955640 1 -- 172.21.15.133:0/2781166876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 msgr2=0x7fc3cc0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:16.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.976+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2781166876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 0x7fc3cc0ca640 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc06d390 tx=0x7fc3c002b910 comp rx=0 tx=0).stop 2024-02-21T23:53:16.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.976+0000 7fc3d1955640 1 -- 172.21.15.133:0/2781166876 shutdown_connections 2024-02-21T23:53:16.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.976+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2781166876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 0x7fc3cc0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:16.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.976+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2781166876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 0x7fc3cc0ca640 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:16.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.976+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2781166876 >> v2:172.21.15.133:3300/0 conn(0x7fc3cc154600 0x7fc3cc0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:16.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.976+0000 7fc3d1955640 1 -- 172.21.15.133:0/2781166876 >> 172.21.15.133:0/2781166876 conn(0x7fc3cc062000 msgr2=0x7fc3cc0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.976+0000 7fc3d1955640 1 -- 172.21.15.133:0/2781166876 shutdown_connections 2024-02-21T23:53:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.976+0000 7fc3d1955640 1 -- 172.21.15.133:0/2781166876 wait complete. 2024-02-21T23:53:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.977+0000 7fc3d1955640 1 Processor -- start 2024-02-21T23:53:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.978+0000 7fc3d1955640 1 -- start start 2024-02-21T23:53:16.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.978+0000 7fc3d1955640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 0x7fc3cc147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:16.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.978+0000 7fc3d1955640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 0x7fc3cc0c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:16.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.978+0000 7fc3d1955640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3cc154600 0x7fc3cc152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:16.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.978+0000 7fc3d1955640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0b5720 con 0x7fc3cc154600 2024-02-21T23:53:16.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.978+0000 7fc3d1955640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0cb860 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.978+0000 7fc3d1955640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0b58a0 con 0x7fc3cc0cab80 2024-02-21T23:53:16.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.979+0000 7fc3caffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 0x7fc3cc147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:16.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.979+0000 7fc3ca7fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 0x7fc3cc0c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:16.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.979+0000 7fc3cb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3cc154600 0x7fc3cc152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:16.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.979+0000 7fc3caffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 0x7fc3cc147f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:52414/0 (socket says 172.21.15.133:52414) 2024-02-21T23:53:16.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.979+0000 7fc3ca7fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 0x7fc3cc0c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:57674/0 (socket says 172.21.15.133:57674) 2024-02-21T23:53:16.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.979+0000 7fc3caffd640 1 -- 172.21.15.133:0/2828760884 learned_addr learned my addr 172.21.15.133:0/2828760884 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:16.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.982+0000 7fc3caffd640 1 -- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 msgr2=0x7fc3cc0c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:16.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.982+0000 7fc3caffd640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 0x7fc3cc0c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:16.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.982+0000 7fc3caffd640 1 -- 172.21.15.133:0/2828760884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3cc154600 msgr2=0x7fc3cc152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:16.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.982+0000 7fc3caffd640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3cc154600 0x7fc3cc152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:16.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.983+0000 7fc3caffd640 1 -- 172.21.15.133:0/2828760884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc1484a0 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.983+0000 7fc3cb7fe640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3cc154600 0x7fc3cc152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:53:16.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.985+0000 7fc3caffd640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 0x7fc3cc147f60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009db0 tx=0x7fc3bc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:16.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.988+0000 7fc3b3fff640 1 -- 172.21.15.133:0/2828760884 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc018020 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.989+0000 7fc3b3fff640 1 -- 172.21.15.133:0/2828760884 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc014480 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.989+0000 7fc3b3fff640 1 -- 172.21.15.133:0/2828760884 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc014850 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.989+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc1f4080 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.989+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc1f32c0 con 0x7fc3cc0c6f50 2024-02-21T23:53:16.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:16.989+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc06a130 con 0x7fc3cc0c6f50 2024-02-21T23:53:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.009+0000 7fc3b3fff640 1 -- 172.21.15.133:0/2828760884 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3bc005d10 con 0x7fc3cc0c6f50 2024-02-21T23:53:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.010+0000 7fc3b3fff640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc3b806b520 0x7fc3b806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:17.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.010+0000 7fc3ca7fc640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc3b806b520 0x7fc3b806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:17.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.010+0000 7fc3ca7fc640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc3b806b520 0x7fc3b806d9e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0036000 tx=0x7fc3c003c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:17.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.012+0000 7fc3b3fff640 1 -- 172.21.15.133:0/2828760884 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fc3bc057cd0 con 0x7fc3cc0c6f50 2024-02-21T23:53:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.260+0000 7fc3b3fff640 1 -- 172.21.15.133:0/2828760884 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fc3bc022070 con 0x7fc3cc0c6f50 2024-02-21T23:53:17.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.465+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3cc157d10 con 0x7fc3cc0c6f50 2024-02-21T23:53:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.504+0000 7fc3b3fff640 1 -- 172.21.15.133:0/2828760884 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19091 (secure 0 0 0) 0x7fc3cc1484a0 con 0x7fc3cc0c6f50 2024-02-21T23:53:17.499 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:53:17.499 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:57.029812+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":0}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:53:17.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.506+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc3b806b520 msgr2=0x7fc3b806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.506+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc3b806b520 0x7fc3b806d9e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0036000 tx=0x7fc3c003c040 comp rx=0 tx=0).stop 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.506+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 msgr2=0x7fc3cc147f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.506+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 0x7fc3cc147f60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009db0 tx=0x7fc3bc00c490 comp rx=0 tx=0).stop 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.506+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 shutdown_connections 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.506+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc3b806b520 0x7fc3b806d9e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.506+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3cc154600 0x7fc3cc152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.506+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3cc0cab80 0x7fc3cc0c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.506+0000 7fc3d1955640 1 --2- 172.21.15.133:0/2828760884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3cc0c6f50 0x7fc3cc147f60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.507+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 >> 172.21.15.133:0/2828760884 conn(0x7fc3cc062000 msgr2=0x7fc3cc149e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.507+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 shutdown_connections 2024-02-21T23:53:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.507+0000 7fc3d1955640 1 -- 172.21.15.133:0/2828760884 wait complete. 2024-02-21T23:53:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-21T23:25:02.902698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-21T23:25:13.993803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-21T23:25:15.604077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-21T23:25:19.468744+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-21T23:53:17.513 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-21T23:53:17.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.621+0000 7f4267faa640 1 Processor -- start 2024-02-21T23:53:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.622+0000 7f4267faa640 1 -- start start 2024-02-21T23:53:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.622+0000 7f4267faa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600cb5f0 0x7f42600cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.622+0000 7f4267faa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f42600c75e0 0x7f42600c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.622+0000 7f4267faa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42600c7f70 con 0x7f4260150f80 2024-02-21T23:53:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.622+0000 7f4267faa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42600c2b40 con 0x7f42600c75e0 2024-02-21T23:53:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.622+0000 7f4267faa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42600c2d70 con 0x7f42600cb5f0 2024-02-21T23:53:17.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.623+0000 7f4265545640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600cb5f0 0x7f42600cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:17.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.623+0000 7f4266547640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f42600c75e0 0x7f42600c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:17.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.623+0000 7f4265d46640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4260150f80 0x7f426015aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43968/0 (socket says 172.21.15.133:43968) 2024-02-21T23:53:17.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.623+0000 7f4265d46640 1 -- 172.21.15.133:0/2498159555 learned_addr learned my addr 172.21.15.133:0/2498159555 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:17.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.626+0000 7f4264d44640 1 -- 172.21.15.133:0/2498159555 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854700641 0 0) 0x7f42600c7f70 con 0x7f4260150f80 2024-02-21T23:53:17.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.626+0000 7f4265545640 1 -- 172.21.15.133:0/2498159555 >> v2:172.21.15.142:3300/0 conn(0x7f42600c75e0 msgr2=0x7f42600c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:17.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.626+0000 7f4265545640 1 --2- 172.21.15.133:0/2498159555 >> v2:172.21.15.142:3300/0 conn(0x7f42600c75e0 0x7f42600c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:17.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.626+0000 7f4265545640 1 -- 172.21.15.133:0/2498159555 >> v1:172.21.15.133:6789/0 conn(0x7f4260150f80 legacy=0x7f426015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:17.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.626+0000 7f4265545640 1 -- 172.21.15.133:0/2498159555 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600c3050 con 0x7f42600cb5f0 2024-02-21T23:53:17.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.628+0000 7f4265545640 1 --2- 172.21.15.133:0/2498159555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600cb5f0 0x7f42600cba10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42540097a0 tx=0x7f425402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7147d03c80ee25 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:17.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4264d44640 1 -- 172.21.15.133:0/2498159555 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254034070 con 0x7f42600cb5f0 2024-02-21T23:53:17.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4264d44640 1 -- 172.21.15.133:0/2498159555 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4254030610 con 0x7f42600cb5f0 2024-02-21T23:53:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4264d44640 1 -- 172.21.15.133:0/2498159555 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540309e0 con 0x7f42600cb5f0 2024-02-21T23:53:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4267faa640 1 -- 172.21.15.133:0/2498159555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600cb5f0 msgr2=0x7f42600cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4267faa640 1 --2- 172.21.15.133:0/2498159555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600cb5f0 0x7f42600cba10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42540097a0 tx=0x7f425402b960 comp rx=0 tx=0).stop 2024-02-21T23:53:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4267faa640 1 -- 172.21.15.133:0/2498159555 shutdown_connections 2024-02-21T23:53:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4267faa640 1 --2- 172.21.15.133:0/2498159555 >> v2:172.21.15.142:3300/0 conn(0x7f42600c75e0 0x7f42600c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:17.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4267faa640 1 --2- 172.21.15.133:0/2498159555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600cb5f0 0x7f42600cba10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:17.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4267faa640 1 -- 172.21.15.133:0/2498159555 >> 172.21.15.133:0/2498159555 conn(0x7f426006aec0 msgr2=0x7f426006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:17.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4267faa640 1 -- 172.21.15.133:0/2498159555 shutdown_connections 2024-02-21T23:53:17.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.632+0000 7f4267faa640 1 -- 172.21.15.133:0/2498159555 wait complete. 2024-02-21T23:53:17.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.633+0000 7f4267faa640 1 Processor -- start 2024-02-21T23:53:17.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.633+0000 7f4267faa640 1 -- start start 2024-02-21T23:53:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.634+0000 7f4267faa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600c75e0 0x7f426013b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.634+0000 7f4267faa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42600cb5f0 0x7f426013bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.634+0000 7f4267faa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4260150f80 0x7f426014b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.634+0000 7f4267faa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426015a5c0 con 0x7f4260150f80 2024-02-21T23:53:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.634+0000 7f4267faa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42600c3610 con 0x7f42600cb5f0 2024-02-21T23:53:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.634+0000 7f4267faa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42600c5990 con 0x7f42600c75e0 2024-02-21T23:53:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.635+0000 7f4265d46640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600c75e0 0x7f426013b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.635+0000 7f4265545640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42600cb5f0 0x7f426013bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.635+0000 7f4266547640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4260150f80 0x7f426014b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.635+0000 7f4265d46640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600c75e0 0x7f426013b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:57688/0 (socket says 172.21.15.133:57688) 2024-02-21T23:53:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.635+0000 7f4265d46640 1 -- 172.21.15.133:0/346042468 learned_addr learned my addr 172.21.15.133:0/346042468 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:17.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.638+0000 7f4265d46640 1 -- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42600cb5f0 msgr2=0x7f426013bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:17.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.638+0000 7f4265d46640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42600cb5f0 0x7f426013bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:17.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.638+0000 7f4265d46640 1 -- 172.21.15.133:0/346042468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4260150f80 msgr2=0x7f426014b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:17.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.638+0000 7f4265d46640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4260150f80 0x7f426014b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:17.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.638+0000 7f4265d46640 1 -- 172.21.15.133:0/346042468 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426013cf20 con 0x7f42600c75e0 2024-02-21T23:53:17.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.641+0000 7f4265d46640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600c75e0 0x7f426013b790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f425800c7f0 tx=0x7f425800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:17.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.644+0000 7f425e7fc640 1 -- 172.21.15.133:0/346042468 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258018070 con 0x7f42600c75e0 2024-02-21T23:53:17.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.644+0000 7f425e7fc640 1 -- 172.21.15.133:0/346042468 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425800a640 con 0x7f42600c75e0 2024-02-21T23:53:17.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.644+0000 7f425e7fc640 1 -- 172.21.15.133:0/346042468 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425800aa30 con 0x7f42600c75e0 2024-02-21T23:53:17.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.644+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426013d1c0 con 0x7f42600c75e0 2024-02-21T23:53:17.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.644+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426013d780 con 0x7f42600c75e0 2024-02-21T23:53:17.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.645+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426006a130 con 0x7f42600c75e0 2024-02-21T23:53:17.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.905+0000 7f425e7fc640 1 -- 172.21.15.133:0/346042468 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f425801b070 con 0x7f42600c75e0 2024-02-21T23:53:17.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.905+0000 7f425e7fc640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f424c06b520 0x7f424c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:17.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.906+0000 7f4265545640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f424c06b520 0x7f424c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:17.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.906+0000 7f4265545640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f424c06b520 0x7f424c06d9e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4254036000 tx=0x7f425403b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:17.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.910+0000 7f425e7fc640 1 -- 172.21.15.133:0/346042468 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f42580565c0 con 0x7f42600c75e0 2024-02-21T23:53:17.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:17.921+0000 7f425e7fc640 1 -- 172.21.15.133:0/346042468 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f4258022020 con 0x7f42600c75e0 2024-02-21T23:53:18.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.095+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f426006b700 con 0x7f42600c75e0 2024-02-21T23:53:18.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.101+0000 7f425e7fc640 1 -- 172.21.15.133:0/346042468 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4258058520 con 0x7f42600c75e0 2024-02-21T23:53:18.094 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-02-21T23:53:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f424c06b520 msgr2=0x7f424c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f424c06b520 0x7f424c06d9e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4254036000 tx=0x7f425403b040 comp rx=0 tx=0).stop 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600c75e0 msgr2=0x7f426013b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600c75e0 0x7f426013b790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f425800c7f0 tx=0x7f425800ccb0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 shutdown_connections 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f424c06b520 0x7f424c06d9e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4260150f80 0x7f426014b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42600cb5f0 0x7f426013bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 --2- 172.21.15.133:0/346042468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42600c75e0 0x7f426013b790 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 >> 172.21.15.133:0/346042468 conn(0x7f426006aec0 msgr2=0x7f42601620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 shutdown_connections 2024-02-21T23:53:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.102+0000 7f4267faa640 1 -- 172.21.15.133:0/346042468 wait complete. 2024-02-21T23:53:18.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-21T23:53:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.218+0000 7f8e77d88640 1 Processor -- start 2024-02-21T23:53:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.218+0000 7f8e77d88640 1 -- start start 2024-02-21T23:53:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.219+0000 7f8e77d88640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8e70154300 0x7f8e7006d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.219+0000 7f8e77d88640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8e700b0fe0 0x7f8e70072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.219+0000 7f8e77d88640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7015afd0 con 0x7f8e70072f80 2024-02-21T23:53:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.219+0000 7f8e77d88640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e7015a170 con 0x7f8e70154300 2024-02-21T23:53:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.219+0000 7f8e77d88640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e7015a450 con 0x7f8e700b0fe0 2024-02-21T23:53:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.220+0000 7f8e75b24640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8e70154300 0x7f8e7006d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.220+0000 7f8e75323640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8e700b0fe0 0x7f8e70072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.220+0000 7f8e75b24640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8e70154300 0x7f8e7006d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59266/0 (socket says 172.21.15.133:59266) 2024-02-21T23:53:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.220+0000 7f8e75b24640 1 -- 172.21.15.133:0/2339770674 learned_addr learned my addr 172.21.15.133:0/2339770674 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.222+0000 7f8e75b24640 1 -- 172.21.15.133:0/2339770674 >> v2:172.21.15.142:3300/0 conn(0x7f8e700b0fe0 msgr2=0x7f8e70072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.222+0000 7f8e75b24640 1 --2- 172.21.15.133:0/2339770674 >> v2:172.21.15.142:3300/0 conn(0x7f8e700b0fe0 0x7f8e70072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.222+0000 7f8e75b24640 1 -- 172.21.15.133:0/2339770674 >> v1:172.21.15.142:6789/0 conn(0x7f8e70072f80 legacy=0x7f8e7006de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.222+0000 7f8e75b24640 1 -- 172.21.15.133:0/2339770674 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7015a730 con 0x7f8e70154300 2024-02-21T23:53:18.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.224+0000 7f8e75b24640 1 --2- 172.21.15.133:0/2339770674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70154300 0x7f8e7006d0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e70061970 tx=0x7f8e6c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d31856147cd926 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:18.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.228+0000 7f8e74b22640 1 -- 172.21.15.133:0/2339770674 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c02cbb0 con 0x7f8e70154300 2024-02-21T23:53:18.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.228+0000 7f8e74b22640 1 -- 172.21.15.133:0/2339770674 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c02cd50 con 0x7f8e70154300 2024-02-21T23:53:18.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.228+0000 7f8e74b22640 1 -- 172.21.15.133:0/2339770674 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c034610 con 0x7f8e70154300 2024-02-21T23:53:18.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.229+0000 7f8e77d88640 1 -- 172.21.15.133:0/2339770674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70154300 msgr2=0x7f8e7006d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.229+0000 7f8e77d88640 1 --2- 172.21.15.133:0/2339770674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70154300 0x7f8e7006d0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e70061970 tx=0x7f8e6c02c010 comp rx=0 tx=0).stop 2024-02-21T23:53:18.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.229+0000 7f8e77d88640 1 -- 172.21.15.133:0/2339770674 shutdown_connections 2024-02-21T23:53:18.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.229+0000 7f8e77d88640 1 --2- 172.21.15.133:0/2339770674 >> v2:172.21.15.142:3300/0 conn(0x7f8e700b0fe0 0x7f8e70072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.229+0000 7f8e77d88640 1 --2- 172.21.15.133:0/2339770674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70154300 0x7f8e7006d0a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.229+0000 7f8e77d88640 1 -- 172.21.15.133:0/2339770674 >> 172.21.15.133:0/2339770674 conn(0x7f8e700cc620 msgr2=0x7f8e700c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:18.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.229+0000 7f8e77d88640 1 -- 172.21.15.133:0/2339770674 shutdown_connections 2024-02-21T23:53:18.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.229+0000 7f8e77d88640 1 -- 172.21.15.133:0/2339770674 wait complete. 2024-02-21T23:53:18.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.230+0000 7f8e77d88640 1 Processor -- start 2024-02-21T23:53:18.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.230+0000 7f8e77d88640 1 -- start start 2024-02-21T23:53:18.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.231+0000 7f8e77d88640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70072f80 0x7f8e701537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.231+0000 7f8e77d88640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e700b0fe0 0x7f8e70145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.231+0000 7f8e77d88640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70154300 0x7f8e701f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.231+0000 7f8e77d88640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e701546e0 con 0x7f8e700b0fe0 2024-02-21T23:53:18.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.231+0000 7f8e77d88640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7015acf0 con 0x7f8e70072f80 2024-02-21T23:53:18.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.231+0000 7f8e77d88640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7006d5e0 con 0x7f8e70154300 2024-02-21T23:53:18.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.231+0000 7f8e75323640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e700b0fe0 0x7f8e70145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.231+0000 7f8e75b24640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70072f80 0x7f8e701537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.231+0000 7f8e76325640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70154300 0x7f8e701f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.232+0000 7f8e75b24640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70072f80 0x7f8e701537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:52448/0 (socket says 172.21.15.133:52448) 2024-02-21T23:53:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.232+0000 7f8e75b24640 1 -- 172.21.15.133:0/3701068008 learned_addr learned my addr 172.21.15.133:0/3701068008 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:18.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.235+0000 7f8e75b24640 1 -- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70154300 msgr2=0x7f8e701f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.235+0000 7f8e75b24640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70154300 0x7f8e701f30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.235+0000 7f8e75b24640 1 -- 172.21.15.133:0/3701068008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e700b0fe0 msgr2=0x7f8e70145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.235+0000 7f8e75b24640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e700b0fe0 0x7f8e70145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.235+0000 7f8e75b24640 1 -- 172.21.15.133:0/3701068008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e701597c0 con 0x7f8e70072f80 2024-02-21T23:53:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.237+0000 7f8e75b24640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70072f80 0x7f8e701537f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c000f80 tx=0x7f8e6c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:18.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.241+0000 7f8e667fc640 1 -- 172.21.15.133:0/3701068008 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c043020 con 0x7f8e70072f80 2024-02-21T23:53:18.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.241+0000 7f8e667fc640 1 -- 172.21.15.133:0/3701068008 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c03e450 con 0x7f8e70072f80 2024-02-21T23:53:18.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.241+0000 7f8e667fc640 1 -- 172.21.15.133:0/3701068008 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c009510 con 0x7f8e70072f80 2024-02-21T23:53:18.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.241+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e70159a30 con 0x7f8e70072f80 2024-02-21T23:53:18.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.241+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e70151db0 con 0x7f8e70072f80 2024-02-21T23:53:18.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.242+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7006a130 con 0x7f8e70072f80 2024-02-21T23:53:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.492+0000 7f8e667fc640 1 -- 172.21.15.133:0/3701068008 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e6c046020 con 0x7f8e70072f80 2024-02-21T23:53:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.492+0000 7f8e667fc640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8e5406b550 0x7f8e5406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.492+0000 7f8e75323640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8e5406b550 0x7f8e5406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.493+0000 7f8e75323640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8e5406b550 0x7f8e5406da10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8e70159080 tx=0x7f8e60009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:18.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.495+0000 7f8e667fc640 1 -- 172.21.15.133:0/3701068008 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f8e6c079630 con 0x7f8e70072f80 2024-02-21T23:53:18.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.506+0000 7f8e667fc640 1 -- 172.21.15.133:0/3701068008 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f8e6c04d070 con 0x7f8e70072f80 2024-02-21T23:53:18.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.681+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e7006c880 con 0x7f8e70072f80 2024-02-21T23:53:18.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.687+0000 7f8e667fc640 1 -- 172.21.15.133:0/3701068008 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e6c07b590 con 0x7f8e70072f80 2024-02-21T23:53:18.681 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8e5406b550 msgr2=0x7f8e5406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8e5406b550 0x7f8e5406da10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8e70159080 tx=0x7f8e60009190 comp rx=0 tx=0).stop 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70072f80 msgr2=0x7f8e701537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70072f80 0x7f8e701537f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c000f80 tx=0x7f8e6c004a40 comp rx=0 tx=0).stop 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 shutdown_connections 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f8e5406b550 0x7f8e5406da10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e70154300 0x7f8e701f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e700b0fe0 0x7f8e70145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 --2- 172.21.15.133:0/3701068008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70072f80 0x7f8e701537f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 >> 172.21.15.133:0/3701068008 conn(0x7f8e700cc620 msgr2=0x7f8e70071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 shutdown_connections 2024-02-21T23:53:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.689+0000 7f8e77d88640 1 -- 172.21.15.133:0/3701068008 wait complete. 2024-02-21T23:53:18.694 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-21T23:53:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.806+0000 7fab65405640 1 Processor -- start 2024-02-21T23:53:18.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.806+0000 7fab65405640 1 -- start start 2024-02-21T23:53:18.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.806+0000 7fab65405640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fab60150a30 0x7fab600c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.806+0000 7fab65405640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab60071f60 0x7fab600c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.806+0000 7fab65405640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab600c53e0 con 0x7fab60071150 2024-02-21T23:53:18.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.806+0000 7fab65405640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab60072540 con 0x7fab60071f60 2024-02-21T23:53:18.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.806+0000 7fab65405640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab600c5680 con 0x7fab60150a30 2024-02-21T23:53:18.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.807+0000 7fab5effd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fab60150a30 0x7fab600c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.807+0000 7fab57fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab60071f60 0x7fab600c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.807+0000 7fab5effd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fab60150a30 0x7fab600c2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:52450/0 (socket says 172.21.15.133:52450) 2024-02-21T23:53:18.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.807+0000 7fab5effd640 1 -- 172.21.15.133:0/1364764455 learned_addr learned my addr 172.21.15.133:0/1364764455 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:18.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.809+0000 7fab5effd640 1 -- 172.21.15.133:0/1364764455 >> v2:172.21.15.133:3300/0 conn(0x7fab60071f60 msgr2=0x7fab600c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:18.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.809+0000 7fab5effd640 1 --2- 172.21.15.133:0/1364764455 >> v2:172.21.15.133:3300/0 conn(0x7fab60071f60 0x7fab600c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.809+0000 7fab5effd640 1 -- 172.21.15.133:0/1364764455 >> v1:172.21.15.133:6789/0 conn(0x7fab60071150 legacy=0x7fab6006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.809+0000 7fab5effd640 1 -- 172.21.15.133:0/1364764455 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600c5960 con 0x7fab60150a30 2024-02-21T23:53:18.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.811+0000 7fab5effd640 1 --2- 172.21.15.133:0/1364764455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab60150a30 0x7fab600c2b40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab6006be20 tx=0x7fab5002e020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9bb22ff33044c2f server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.816+0000 7fab5e7fc640 1 -- 172.21.15.133:0/1364764455 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5003b070 con 0x7fab60150a30 2024-02-21T23:53:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.816+0000 7fab5e7fc640 1 -- 172.21.15.133:0/1364764455 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5002edd0 con 0x7fab60150a30 2024-02-21T23:53:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.816+0000 7fab5e7fc640 1 -- 172.21.15.133:0/1364764455 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000b040 con 0x7fab60150a30 2024-02-21T23:53:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.817+0000 7fab65405640 1 -- 172.21.15.133:0/1364764455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab60150a30 msgr2=0x7fab600c2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.817+0000 7fab65405640 1 --2- 172.21.15.133:0/1364764455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab60150a30 0x7fab600c2b40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab6006be20 tx=0x7fab5002e020 comp rx=0 tx=0).stop 2024-02-21T23:53:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.817+0000 7fab65405640 1 -- 172.21.15.133:0/1364764455 shutdown_connections 2024-02-21T23:53:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.817+0000 7fab65405640 1 --2- 172.21.15.133:0/1364764455 >> v2:172.21.15.133:3300/0 conn(0x7fab60071f60 0x7fab600c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.817+0000 7fab65405640 1 --2- 172.21.15.133:0/1364764455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab60150a30 0x7fab600c2b40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.817+0000 7fab65405640 1 -- 172.21.15.133:0/1364764455 >> 172.21.15.133:0/1364764455 conn(0x7fab6006aec0 msgr2=0x7fab6006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.817+0000 7fab65405640 1 -- 172.21.15.133:0/1364764455 shutdown_connections 2024-02-21T23:53:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.817+0000 7fab65405640 1 -- 172.21.15.133:0/1364764455 wait complete. 2024-02-21T23:53:18.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.818+0000 7fab65405640 1 Processor -- start 2024-02-21T23:53:18.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.818+0000 7fab65405640 1 -- start start 2024-02-21T23:53:18.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab65405640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab60071150 0x7fab601eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab65405640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab60071f60 0x7fab600ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab65405640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab60150a30 0x7fab6014ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:18.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab65405640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab60150e10 con 0x7fab60150a30 2024-02-21T23:53:18.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab65405640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab60072a70 con 0x7fab60071f60 2024-02-21T23:53:18.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab65405640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab60150f90 con 0x7fab60071150 2024-02-21T23:53:18.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab5f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab60150a30 0x7fab6014ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab5effd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab60071150 0x7fab601eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab57fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab60071f60 0x7fab600ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:18.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.819+0000 7fab5effd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab60071150 0x7fab601eb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:57714/0 (socket says 172.21.15.133:57714) 2024-02-21T23:53:18.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.820+0000 7fab5effd640 1 -- 172.21.15.133:0/2543803276 learned_addr learned my addr 172.21.15.133:0/2543803276 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:18.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.823+0000 7fab5f7fe640 1 -- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab60071150 msgr2=0x7fab601eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.823+0000 7fab5f7fe640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab60071150 0x7fab601eb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.823+0000 7fab5f7fe640 1 -- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab60071f60 msgr2=0x7fab600ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:18.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.823+0000 7fab5f7fe640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab60071f60 0x7fab600ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:18.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.823+0000 7fab5f7fe640 1 -- 172.21.15.133:0/2543803276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab60143090 con 0x7fab60150a30 2024-02-21T23:53:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.825+0000 7fab5f7fe640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab60150a30 0x7fab6014ed60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c6c0 tx=0x7fab5800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:18.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.829+0000 7fab577fe640 1 -- 172.21.15.133:0/2543803276 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58003b70 con 0x7fab60150a30 2024-02-21T23:53:18.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.829+0000 7fab577fe640 1 -- 172.21.15.133:0/2543803276 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab58003d10 con 0x7fab60150a30 2024-02-21T23:53:18.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.829+0000 7fab577fe640 1 -- 172.21.15.133:0/2543803276 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab580052f0 con 0x7fab60150a30 2024-02-21T23:53:18.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.829+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab60143370 con 0x7fab60150a30 2024-02-21T23:53:18.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.829+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab60143930 con 0x7fab60150a30 2024-02-21T23:53:18.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:18.830+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab600c37b0 con 0x7fab60150a30 2024-02-21T23:53:19.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.076+0000 7fab577fe640 1 -- 172.21.15.133:0/2543803276 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab58005490 con 0x7fab60150a30 2024-02-21T23:53:19.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.076+0000 7fab577fe640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab4006b4d0 0x7fab4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:19.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.077+0000 7fab5effd640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab4006b4d0 0x7fab4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:19.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.077+0000 7fab5effd640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab4006b4d0 0x7fab4006d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab5002e060 tx=0x7fab5000c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:19.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.090+0000 7fab577fe640 1 -- 172.21.15.133:0/2543803276 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fab580572b0 con 0x7fab60150a30 2024-02-21T23:53:19.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.090+0000 7fab577fe640 1 -- 172.21.15.133:0/2543803276 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fab600c37b0 con 0x7fab60150a30 2024-02-21T23:53:19.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.262+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab6006a130 con 0x7fab60150a30 2024-02-21T23:53:19.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.269+0000 7fab577fe640 1 -- 172.21.15.133:0/2543803276 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab58059210 con 0x7fab60150a30 2024-02-21T23:53:19.263 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-02-21T23:53:19.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab4006b4d0 msgr2=0x7fab4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab4006b4d0 0x7fab4006d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab5002e060 tx=0x7fab5000c040 comp rx=0 tx=0).stop 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab60150a30 msgr2=0x7fab6014ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab60150a30 0x7fab6014ed60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c6c0 tx=0x7fab5800cb80 comp rx=0 tx=0).stop 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 shutdown_connections 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fab4006b4d0 0x7fab4006d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab60150a30 0x7fab6014ed60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab60071f60 0x7fab600ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 --2- 172.21.15.133:0/2543803276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab60071150 0x7fab601eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 >> 172.21.15.133:0/2543803276 conn(0x7fab6006aec0 msgr2=0x7fab60071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 shutdown_connections 2024-02-21T23:53:19.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.271+0000 7fab65405640 1 -- 172.21.15.133:0/2543803276 wait complete. 2024-02-21T23:53:19.277 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-21T23:53:19.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.387+0000 7f9903aa2640 1 Processor -- start 2024-02-21T23:53:19.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.387+0000 7f9903aa2640 1 -- start start 2024-02-21T23:53:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.387+0000 7f9903aa2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc150f80 0x7f98fc15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.388+0000 7f9903aa2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f98fc0c75e0 0x7f98fc0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.388+0000 7f9903aa2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc0c7f70 con 0x7f98fc15b020 2024-02-21T23:53:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.388+0000 7f9903aa2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc0c2b40 con 0x7f98fc0c75e0 2024-02-21T23:53:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.388+0000 7f9903aa2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0c2d70 con 0x7f98fc150f80 2024-02-21T23:53:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.388+0000 7f990183e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc150f80 0x7f98fc15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.388+0000 7f990183e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc150f80 0x7f98fc15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:57730/0 (socket says 172.21.15.133:57730) 2024-02-21T23:53:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.389+0000 7f990183e640 1 -- 172.21.15.133:0/3349735023 learned_addr learned my addr 172.21.15.133:0/3349735023 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.389+0000 7f990203f640 1 --2- 172.21.15.133:0/3349735023 >> v2:172.21.15.142:3300/0 conn(0x7f98fc0c75e0 0x7f98fc0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:19.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.391+0000 7f990183e640 1 -- 172.21.15.133:0/3349735023 >> v2:172.21.15.142:3300/0 conn(0x7f98fc0c75e0 msgr2=0x7f98fc0c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:19.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.391+0000 7f990183e640 1 --2- 172.21.15.133:0/3349735023 >> v2:172.21.15.142:3300/0 conn(0x7f98fc0c75e0 0x7f98fc0c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.391+0000 7f990183e640 1 -- 172.21.15.133:0/3349735023 >> v1:172.21.15.142:6789/0 conn(0x7f98fc15b020 legacy=0x7f98fc0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:19.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.391+0000 7f990183e640 1 -- 172.21.15.133:0/3349735023 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc0c3050 con 0x7f98fc150f80 2024-02-21T23:53:19.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.393+0000 7f990183e640 1 --2- 172.21.15.133:0/3349735023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc150f80 0x7f98fc15aae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98f800bb10 tx=0x7f98f802e1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4257f75fb104aac1 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.397+0000 7f990083c640 1 -- 172.21.15.133:0/3349735023 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f803b070 con 0x7f98fc150f80 2024-02-21T23:53:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f990083c640 1 -- 172.21.15.133:0/3349735023 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f8036440 con 0x7f98fc150f80 2024-02-21T23:53:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f990083c640 1 -- 172.21.15.133:0/3349735023 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8036810 con 0x7f98fc150f80 2024-02-21T23:53:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f9903aa2640 1 -- 172.21.15.133:0/3349735023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc150f80 msgr2=0x7f98fc15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f9903aa2640 1 --2- 172.21.15.133:0/3349735023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc150f80 0x7f98fc15aae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98f800bb10 tx=0x7f98f802e1f0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f9903aa2640 1 -- 172.21.15.133:0/3349735023 shutdown_connections 2024-02-21T23:53:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f9903aa2640 1 --2- 172.21.15.133:0/3349735023 >> v2:172.21.15.142:3300/0 conn(0x7f98fc0c75e0 0x7f98fc0c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f9903aa2640 1 --2- 172.21.15.133:0/3349735023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc150f80 0x7f98fc15aae0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f9903aa2640 1 -- 172.21.15.133:0/3349735023 >> 172.21.15.133:0/3349735023 conn(0x7f98fc06aec0 msgr2=0x7f98fc06b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f9903aa2640 1 -- 172.21.15.133:0/3349735023 shutdown_connections 2024-02-21T23:53:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.398+0000 7f9903aa2640 1 -- 172.21.15.133:0/3349735023 wait complete. 2024-02-21T23:53:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.399+0000 7f9903aa2640 1 Processor -- start 2024-02-21T23:53:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.399+0000 7f9903aa2640 1 -- start start 2024-02-21T23:53:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.400+0000 7f9903aa2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc0c75e0 0x7f98fc150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.400+0000 7f9903aa2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc150f80 0x7f98fc1487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.400+0000 7f9903aa2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc15b020 0x7f98fc1ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.400+0000 7f9903aa2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15a5c0 con 0x7f98fc15b020 2024-02-21T23:53:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.400+0000 7f9903aa2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0c3610 con 0x7f98fc150f80 2024-02-21T23:53:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.400+0000 7f9903aa2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0c5990 con 0x7f98fc0c75e0 2024-02-21T23:53:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.400+0000 7f990183e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc0c75e0 0x7f98fc150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.400+0000 7f990103d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc150f80 0x7f98fc1487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.401+0000 7f990203f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc15b020 0x7f98fc1ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.401+0000 7f990183e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc0c75e0 0x7f98fc150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:57744/0 (socket says 172.21.15.133:57744) 2024-02-21T23:53:19.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.401+0000 7f990183e640 1 -- 172.21.15.133:0/21997744 learned_addr learned my addr 172.21.15.133:0/21997744 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.404+0000 7f990183e640 1 -- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc150f80 msgr2=0x7f98fc1487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.404+0000 7f990183e640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc150f80 0x7f98fc1487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.404+0000 7f990183e640 1 -- 172.21.15.133:0/21997744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc15b020 msgr2=0x7f98fc1ef4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.404+0000 7f990183e640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc15b020 0x7f98fc1ef4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.404+0000 7f990183e640 1 -- 172.21.15.133:0/21997744 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc14d320 con 0x7f98fc0c75e0 2024-02-21T23:53:19.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.406+0000 7f990183e640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc0c75e0 0x7f98fc150020 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f98f802e230 tx=0x7f98f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.409+0000 7f98f27fc640 1 -- 172.21.15.133:0/21997744 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8045030 con 0x7f98fc0c75e0 2024-02-21T23:53:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.409+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc1ec370 con 0x7f98fc0c75e0 2024-02-21T23:53:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.409+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc14cef0 con 0x7f98fc0c75e0 2024-02-21T23:53:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.410+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc06a130 con 0x7f98fc0c75e0 2024-02-21T23:53:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.410+0000 7f98f27fc640 1 -- 172.21.15.133:0/21997744 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f8036440 con 0x7f98fc0c75e0 2024-02-21T23:53:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.410+0000 7f98f27fc640 1 -- 172.21.15.133:0/21997744 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f803b040 con 0x7f98fc0c75e0 2024-02-21T23:53:19.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.688+0000 7f98f27fc640 1 -- 172.21.15.133:0/21997744 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98f8048070 con 0x7f98fc0c75e0 2024-02-21T23:53:19.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.688+0000 7f98f27fc640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98e006b500 0x7f98e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:19.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.688+0000 7f990103d640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98e006b500 0x7f98e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:19.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.689+0000 7f990103d640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98e006b500 0x7f98e006d9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f98fc1508d0 tx=0x7f98ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:19.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.692+0000 7f98f27fc640 1 -- 172.21.15.133:0/21997744 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f98f807b710 con 0x7f98fc0c75e0 2024-02-21T23:53:19.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.703+0000 7f98f27fc640 1 -- 172.21.15.133:0/21997744 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f98f804f070 con 0x7f98fc0c75e0 2024-02-21T23:53:19.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.886+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f98fc14f380 con 0x7f98fc0c75e0 2024-02-21T23:53:19.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.928+0000 7f98f27fc640 1 -- 172.21.15.133:0/21997744 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f98f807d670 con 0x7f98fc0c75e0 2024-02-21T23:53:19.922 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:53:19.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98e006b500 msgr2=0x7f98e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:19.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98e006b500 0x7f98e006d9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f98fc1508d0 tx=0x7f98ec009040 comp rx=0 tx=0).stop 2024-02-21T23:53:19.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc0c75e0 msgr2=0x7f98fc150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:19.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc0c75e0 0x7f98fc150020 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f98f802e230 tx=0x7f98f80046c0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 shutdown_connections 2024-02-21T23:53:19.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f98e006b500 0x7f98e006d9c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc15b020 0x7f98fc1ef4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98fc150f80 0x7f98fc1487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 --2- 172.21.15.133:0/21997744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98fc0c75e0 0x7f98fc150020 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:19.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 >> 172.21.15.133:0/21997744 conn(0x7f98fc06aec0 msgr2=0x7f98fc06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:19.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 shutdown_connections 2024-02-21T23:53:19.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:19.930+0000 7f9903aa2640 1 -- 172.21.15.133:0/21997744 wait complete. 2024-02-21T23:53:19.936 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-21T23:53:20.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.042+0000 7fbe071cf640 1 Processor -- start 2024-02-21T23:53:20.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.042+0000 7fbe071cf640 1 -- start start 2024-02-21T23:53:20.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.043+0000 7fbe071cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe000c4fc0 0x7fbe00071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.043+0000 7fbe071cf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbe000c53a0 0x7fbe0006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.043+0000 7fbe071cf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbe0006f700 0x7fbe0015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.043+0000 7fbe071cf640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe00158eb0 con 0x7fbe000c53a0 2024-02-21T23:53:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.043+0000 7fbe071cf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe00158230 con 0x7fbe0006f700 2024-02-21T23:53:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.043+0000 7fbe071cf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00158510 con 0x7fbe000c4fc0 2024-02-21T23:53:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.044+0000 7fbdfffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbe000c53a0 0x7fbe0006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.044+0000 7fbe0576c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbe0006f700 0x7fbe0015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.044+0000 7fbe04f6b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe000c4fc0 0x7fbe00071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.044+0000 7fbe0576c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbe0006f700 0x7fbe0015b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:52484/0 (socket says 172.21.15.133:52484) 2024-02-21T23:53:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.044+0000 7fbe0576c640 1 -- 172.21.15.133:0/1135341518 learned_addr learned my addr 172.21.15.133:0/1135341518 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:20.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.044+0000 7fbe04f6b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe000c4fc0 0x7fbe00071b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:57748/0 (socket says 172.21.15.133:57748) 2024-02-21T23:53:20.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.046+0000 7fbe0576c640 1 -- 172.21.15.133:0/1135341518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe000c4fc0 msgr2=0x7fbe00071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:20.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.046+0000 7fbe0576c640 1 --2- 172.21.15.133:0/1135341518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe000c4fc0 0x7fbe00071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.046+0000 7fbe0576c640 1 -- 172.21.15.133:0/1135341518 >> v2:172.21.15.133:3300/0 conn(0x7fbe000c53a0 msgr2=0x7fbe0006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.046+0000 7fbe0576c640 1 --2- 172.21.15.133:0/1135341518 >> v2:172.21.15.133:3300/0 conn(0x7fbe000c53a0 0x7fbe0006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.046+0000 7fbe0576c640 1 -- 172.21.15.133:0/1135341518 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe001587f0 con 0x7fbe0006f700 2024-02-21T23:53:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.046+0000 7fbe04f6b640 1 --2- 172.21.15.133:0/1135341518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe000c4fc0 0x7fbe00071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:53:20.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.048+0000 7fbe0576c640 1 --2- 172.21.15.133:0/1135341518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe0006f700 0x7fbe0015b7b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe00155af0 tx=0x7fbdf802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93fc28d04fc8e11c server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:20.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbdff7fe640 1 -- 172.21.15.133:0/1135341518 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf802cb00 con 0x7fbe0006f700 2024-02-21T23:53:20.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbdff7fe640 1 -- 172.21.15.133:0/1135341518 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf802cca0 con 0x7fbe0006f700 2024-02-21T23:53:20.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbdff7fe640 1 -- 172.21.15.133:0/1135341518 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8034610 con 0x7fbe0006f700 2024-02-21T23:53:20.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbe071cf640 1 -- 172.21.15.133:0/1135341518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe0006f700 msgr2=0x7fbe0015b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbe071cf640 1 --2- 172.21.15.133:0/1135341518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe0006f700 0x7fbe0015b7b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe00155af0 tx=0x7fbdf802bf60 comp rx=0 tx=0).stop 2024-02-21T23:53:20.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbe071cf640 1 -- 172.21.15.133:0/1135341518 shutdown_connections 2024-02-21T23:53:20.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbe071cf640 1 --2- 172.21.15.133:0/1135341518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe0006f700 0x7fbe0015b7b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbe071cf640 1 --2- 172.21.15.133:0/1135341518 >> v2:172.21.15.133:3300/0 conn(0x7fbe000c53a0 0x7fbe0006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbe071cf640 1 --2- 172.21.15.133:0/1135341518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe000c4fc0 0x7fbe00071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbe071cf640 1 -- 172.21.15.133:0/1135341518 >> 172.21.15.133:0/1135341518 conn(0x7fbe0006aec0 msgr2=0x7fbe0006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:20.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbe071cf640 1 -- 172.21.15.133:0/1135341518 shutdown_connections 2024-02-21T23:53:20.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.052+0000 7fbe071cf640 1 -- 172.21.15.133:0/1135341518 wait complete. 2024-02-21T23:53:20.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.054+0000 7fbe071cf640 1 Processor -- start 2024-02-21T23:53:20.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.054+0000 7fbe071cf640 1 -- start start 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.054+0000 7fbe071cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe0006f700 0x7fbe001ed670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.054+0000 7fbe071cf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe000c4fc0 0x7fbe000caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.054+0000 7fbe071cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe000c53a0 0x7fbe000cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.054+0000 7fbe071cf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0015bcf0 con 0x7fbe000c53a0 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.054+0000 7fbe071cf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00157f20 con 0x7fbe000c4fc0 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.054+0000 7fbe071cf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0015be70 con 0x7fbe0006f700 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.055+0000 7fbdfffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe000c4fc0 0x7fbe000caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.055+0000 7fbe04f6b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe0006f700 0x7fbe001ed670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.055+0000 7fbe0576c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe000c53a0 0x7fbe000cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.055+0000 7fbdfffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe000c4fc0 0x7fbe000caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:52494/0 (socket says 172.21.15.133:52494) 2024-02-21T23:53:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.055+0000 7fbdfffff640 1 -- 172.21.15.133:0/425480029 learned_addr learned my addr 172.21.15.133:0/425480029 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:20.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.058+0000 7fbdfffff640 1 -- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe0006f700 msgr2=0x7fbe001ed670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.058+0000 7fbdfffff640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe0006f700 0x7fbe001ed670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.058+0000 7fbdfffff640 1 -- 172.21.15.133:0/425480029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe000c53a0 msgr2=0x7fbe000cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.058+0000 7fbdfffff640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe000c53a0 0x7fbe000cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.058+0000 7fbdfffff640 1 -- 172.21.15.133:0/425480029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00152f50 con 0x7fbe000c4fc0 2024-02-21T23:53:20.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.061+0000 7fbdfffff640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe000c4fc0 0x7fbe000caa80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400e8a0 tx=0x7fbdf400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.063+0000 7fbdfd7fa640 1 -- 172.21.15.133:0/425480029 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4017070 con 0x7fbe000c4fc0 2024-02-21T23:53:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.063+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe000c9700 con 0x7fbe000c4fc0 2024-02-21T23:53:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.063+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe000c9cc0 con 0x7fbe000c4fc0 2024-02-21T23:53:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.064+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0006a130 con 0x7fbe000c4fc0 2024-02-21T23:53:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.064+0000 7fbdfd7fa640 1 -- 172.21.15.133:0/425480029 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf400c940 con 0x7fbe000c4fc0 2024-02-21T23:53:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.064+0000 7fbdfd7fa640 1 -- 172.21.15.133:0/425480029 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4021060 con 0x7fbe000c4fc0 2024-02-21T23:53:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.319+0000 7fbdfd7fa640 1 -- 172.21.15.133:0/425480029 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdf401a070 con 0x7fbe000c4fc0 2024-02-21T23:53:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.319+0000 7fbdfd7fa640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbdec06b500 0x7fbdec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.319+0000 7fbe04f6b640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbdec06b500 0x7fbdec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.320+0000 7fbe04f6b640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbdec06b500 0x7fbdec06d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009560 tx=0x7fbdf0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.322+0000 7fbdfd7fa640 1 -- 172.21.15.133:0/425480029 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fbdf40570f0 con 0x7fbe000c4fc0 2024-02-21T23:53:20.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.332+0000 7fbdfd7fa640 1 -- 172.21.15.133:0/425480029 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fbdf4059050 con 0x7fbe000c4fc0 2024-02-21T23:53:20.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.517+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbe000c7010 con 0x7fbe000c4fc0 2024-02-21T23:53:20.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.521+0000 7fbdfd7fa640 1 -- 172.21.15.133:0/425480029 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbdf405d0f0 con 0x7fbe000c4fc0 2024-02-21T23:53:20.515 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbdec06b500 msgr2=0x7fbdec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbdec06b500 0x7fbdec06d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009560 tx=0x7fbdf0009190 comp rx=0 tx=0).stop 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe000c4fc0 msgr2=0x7fbe000caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe000c4fc0 0x7fbe000caa80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400e8a0 tx=0x7fbdf400ed60 comp rx=0 tx=0).stop 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 shutdown_connections 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbdec06b500 0x7fbdec06d9c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe000c53a0 0x7fbe000cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe000c4fc0 0x7fbe000caa80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 --2- 172.21.15.133:0/425480029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe0006f700 0x7fbe001ed670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 >> 172.21.15.133:0/425480029 conn(0x7fbe0006aec0 msgr2=0x7fbe000c6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:20.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 shutdown_connections 2024-02-21T23:53:20.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.524+0000 7fbe071cf640 1 -- 172.21.15.133:0/425480029 wait complete. 2024-02-21T23:53:20.529 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-02-21T23:53:20.529 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-21T23:53:20.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:53:20.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.692+0000 7f925cbb4640 1 Processor -- start 2024-02-21T23:53:20.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.692+0000 7f925cbb4640 1 -- start start 2024-02-21T23:53:20.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.693+0000 7f925cbb4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92580cdfe0 0x7f92580c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.693+0000 7f925cbb4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92580717e0 con 0x7f92580ce3c0 2024-02-21T23:53:20.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.693+0000 7f925cbb4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9258158090 con 0x7f925814e510 2024-02-21T23:53:20.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.693+0000 7f925cbb4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92580c84b0 con 0x7f92580cdfe0 2024-02-21T23:53:20.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.694+0000 7f925659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92580cdfe0 0x7f92580c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.694+0000 7f9256d9d640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f925814e510 0x7f925814e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:51098/0 (socket says 172.21.15.133:51098) 2024-02-21T23:53:20.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.694+0000 7f9256d9d640 1 -- 172.21.15.133:0/2062357140 learned_addr learned my addr 172.21.15.133:0/2062357140 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:20.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.696+0000 7f925559a640 1 -- 172.21.15.133:0/2062357140 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651339235 0 0) 0x7f9258158090 con 0x7f925814e510 2024-02-21T23:53:20.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.696+0000 7f925559a640 1 -- 172.21.15.133:0/2062357140 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9240003520 con 0x7f925814e510 2024-02-21T23:53:20.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.697+0000 7f925659c640 1 -- 172.21.15.133:0/2062357140 >> v1:172.21.15.142:6789/0 conn(0x7f925814e510 legacy=0x7f925814e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.697+0000 7f925659c640 1 -- 172.21.15.133:0/2062357140 >> v1:172.21.15.133:6789/0 conn(0x7f92580ce3c0 legacy=0x7f9258070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.697+0000 7f925659c640 1 -- 172.21.15.133:0/2062357140 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9258158370 con 0x7f92580cdfe0 2024-02-21T23:53:20.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.701+0000 7f925659c640 1 --2- 172.21.15.133:0/2062357140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580cdfe0 0x7f92580c3250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92500097a0 tx=0x7f925002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abee040a7efdbf1 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:20.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.704+0000 7f925559a640 1 -- 172.21.15.133:0/2062357140 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250034070 con 0x7f92580cdfe0 2024-02-21T23:53:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.705+0000 7f925559a640 1 -- 172.21.15.133:0/2062357140 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9250030610 con 0x7f92580cdfe0 2024-02-21T23:53:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.705+0000 7f925559a640 1 -- 172.21.15.133:0/2062357140 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500309c0 con 0x7f92580cdfe0 2024-02-21T23:53:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.705+0000 7f925cbb4640 1 -- 172.21.15.133:0/2062357140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580cdfe0 msgr2=0x7f92580c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.705+0000 7f925cbb4640 1 --2- 172.21.15.133:0/2062357140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580cdfe0 0x7f92580c3250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92500097a0 tx=0x7f925002b960 comp rx=0 tx=0).stop 2024-02-21T23:53:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.705+0000 7f925cbb4640 1 -- 172.21.15.133:0/2062357140 shutdown_connections 2024-02-21T23:53:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.705+0000 7f925cbb4640 1 --2- 172.21.15.133:0/2062357140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580cdfe0 0x7f92580c3250 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.705+0000 7f925cbb4640 1 -- 172.21.15.133:0/2062357140 >> 172.21.15.133:0/2062357140 conn(0x7f925806aec0 msgr2=0x7f925806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.706+0000 7f925cbb4640 1 -- 172.21.15.133:0/2062357140 shutdown_connections 2024-02-21T23:53:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.706+0000 7f925cbb4640 1 -- 172.21.15.133:0/2062357140 wait complete. 2024-02-21T23:53:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.707+0000 7f925cbb4640 1 Processor -- start 2024-02-21T23:53:20.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.707+0000 7f925cbb4640 1 -- start start 2024-02-21T23:53:20.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.707+0000 7f925cbb4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92580cdfe0 0x7f92580aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f925cbb4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580ce3c0 0x7f92580b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f925cbb4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f925814e510 0x7f925814b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f925cbb4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9258158930 con 0x7f92580ce3c0 2024-02-21T23:53:20.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f925cbb4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9258158ca0 con 0x7f925814e510 2024-02-21T23:53:20.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f925cbb4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9258071480 con 0x7f92580cdfe0 2024-02-21T23:53:20.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580ce3c0 0x7f92580b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f925659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92580cdfe0 0x7f92580aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f9256d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f925814e510 0x7f925814b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f925659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92580cdfe0 0x7f92580aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:57756/0 (socket says 172.21.15.133:57756) 2024-02-21T23:53:20.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f9255d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580ce3c0 0x7f92580b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59354/0 (socket says 172.21.15.133:59354) 2024-02-21T23:53:20.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.708+0000 7f925659c640 1 -- 172.21.15.133:0/3098466706 learned_addr learned my addr 172.21.15.133:0/3098466706 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:20.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.712+0000 7f9255d9b640 1 -- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92580cdfe0 msgr2=0x7f92580aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.712+0000 7f9255d9b640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92580cdfe0 0x7f92580aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.712+0000 7f9255d9b640 1 -- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f925814e510 msgr2=0x7f925814b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:20.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.712+0000 7f9255d9b640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f925814e510 0x7f925814b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:20.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.712+0000 7f9255d9b640 1 -- 172.21.15.133:0/3098466706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580af9e0 con 0x7f92580ce3c0 2024-02-21T23:53:20.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.714+0000 7f9255d9b640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580ce3c0 0x7f92580b0810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c770 tx=0x7f924c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.717+0000 7f9246ffd640 1 -- 172.21.15.133:0/3098466706 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c003b70 con 0x7f92580ce3c0 2024-02-21T23:53:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.717+0000 7f9246ffd640 1 -- 172.21.15.133:0/3098466706 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c003d10 con 0x7f92580ce3c0 2024-02-21T23:53:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.717+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92580c99a0 con 0x7f92580ce3c0 2024-02-21T23:53:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.717+0000 7f9246ffd640 1 -- 172.21.15.133:0/3098466706 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00aa30 con 0x7f92580ce3c0 2024-02-21T23:53:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.717+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92580adc60 con 0x7f92580ce3c0 2024-02-21T23:53:20.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.718+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925814d6a0 con 0x7f92580ce3c0 2024-02-21T23:53:20.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.968+0000 7f9246ffd640 1 -- 172.21.15.133:0/3098466706 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f924c00abd0 con 0x7f92580ce3c0 2024-02-21T23:53:20.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.969+0000 7f9246ffd640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f923806b520 0x7f923806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:20.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.969+0000 7f925659c640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f923806b520 0x7f923806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:20.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.969+0000 7f925659c640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f923806b520 0x7f923806d9e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9250009560 tx=0x7f9250008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:20.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.983+0000 7f9246ffd640 1 -- 172.21.15.133:0/3098466706 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f924c0572b0 con 0x7f92580ce3c0 2024-02-21T23:53:20.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:20.983+0000 7f9246ffd640 1 -- 172.21.15.133:0/3098466706 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f925814d6a0 con 0x7f92580ce3c0 2024-02-21T23:53:21.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.180+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f925806a130 con 0x7f92580ce3c0 2024-02-21T23:53:21.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.218+0000 7f9246ffd640 1 -- 172.21.15.133:0/3098466706 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19091 (secure 0 0 0) 0x7f92580af9e0 con 0x7f92580ce3c0 2024-02-21T23:53:21.213 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:53:21.213 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:57.029812+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":0}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f923806b520 msgr2=0x7f923806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f923806b520 0x7f923806d9e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9250009560 tx=0x7f9250008030 comp rx=0 tx=0).stop 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580ce3c0 msgr2=0x7f92580b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580ce3c0 0x7f92580b0810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c770 tx=0x7f924c00cc30 comp rx=0 tx=0).stop 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 shutdown_connections 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f923806b520 0x7f923806d9e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f925814e510 0x7f925814b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92580ce3c0 0x7f92580b0810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 --2- 172.21.15.133:0/3098466706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92580cdfe0 0x7f92580aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 >> 172.21.15.133:0/3098466706 conn(0x7f925806aec0 msgr2=0x7f925814ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 shutdown_connections 2024-02-21T23:53:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.221+0000 7f925cbb4640 1 -- 172.21.15.133:0/3098466706 wait complete. 2024-02-21T23:53:21.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-21T23:25:02.902698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-21T23:25:13.993803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-21T23:25:15.604077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-21T23:25:19.468744+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-21T23:53:21.227 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-21T23:53:21.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.336+0000 7fbc32e2d640 1 Processor -- start 2024-02-21T23:53:21.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.336+0000 7fbc32e2d640 1 -- start start 2024-02-21T23:53:21.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.337+0000 7fbc32e2d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c0b0fe0 0x7fbc2c072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.337+0000 7fbc32e2d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c15afd0 con 0x7fbc2c154300 2024-02-21T23:53:21.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.337+0000 7fbc32e2d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c15a170 con 0x7fbc2c072f80 2024-02-21T23:53:21.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.337+0000 7fbc32e2d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c15a450 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.338+0000 7fbc313ca640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fbc2c072f80 0x7fbc2c06de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:51114/0 (socket says 172.21.15.133:51114) 2024-02-21T23:53:21.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.338+0000 7fbc313ca640 1 -- 172.21.15.133:0/3372564235 learned_addr learned my addr 172.21.15.133:0/3372564235 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:21.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.338+0000 7fbc2bfff640 1 --2- 172.21.15.133:0/3372564235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c0b0fe0 0x7fbc2c072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.340+0000 7fbc2b7fe640 1 -- 172.21.15.133:0/3372564235 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122084074 0 0) 0x7fbc2c15a170 con 0x7fbc2c072f80 2024-02-21T23:53:21.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.340+0000 7fbc2b7fe640 1 -- 172.21.15.133:0/3372564235 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc14003520 con 0x7fbc2c072f80 2024-02-21T23:53:21.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.340+0000 7fbc2b7fe640 1 -- 172.21.15.133:0/3372564235 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419450518 0 0) 0x7fbc2c15afd0 con 0x7fbc2c154300 2024-02-21T23:53:21.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.340+0000 7fbc2b7fe640 1 -- 172.21.15.133:0/3372564235 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc2c15a170 con 0x7fbc2c154300 2024-02-21T23:53:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.341+0000 7fbc2bfff640 1 -- 172.21.15.133:0/3372564235 >> v1:172.21.15.142:6789/0 conn(0x7fbc2c072f80 legacy=0x7fbc2c06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.341+0000 7fbc2bfff640 1 -- 172.21.15.133:0/3372564235 >> v1:172.21.15.133:6789/0 conn(0x7fbc2c154300 legacy=0x7fbc2c06d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.341+0000 7fbc2bfff640 1 -- 172.21.15.133:0/3372564235 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c15a730 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.344+0000 7fbc2bfff640 1 --2- 172.21.15.133:0/3372564235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c0b0fe0 0x7fbc2c072a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc200097d0 tx=0x7fbc2002b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f42593d3d11318 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:21.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc2b7fe640 1 -- 172.21.15.133:0/3372564235 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20034070 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc2b7fe640 1 -- 172.21.15.133:0/3372564235 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc20030880 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc2b7fe640 1 -- 172.21.15.133:0/3372564235 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20030c50 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc32e2d640 1 -- 172.21.15.133:0/3372564235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c0b0fe0 msgr2=0x7fbc2c072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc32e2d640 1 --2- 172.21.15.133:0/3372564235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c0b0fe0 0x7fbc2c072a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc200097d0 tx=0x7fbc2002b9c0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc32e2d640 1 -- 172.21.15.133:0/3372564235 shutdown_connections 2024-02-21T23:53:21.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc32e2d640 1 --2- 172.21.15.133:0/3372564235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c0b0fe0 0x7fbc2c072a40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc32e2d640 1 -- 172.21.15.133:0/3372564235 >> 172.21.15.133:0/3372564235 conn(0x7fbc2c0cc620 msgr2=0x7fbc2c0c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:21.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc32e2d640 1 -- 172.21.15.133:0/3372564235 shutdown_connections 2024-02-21T23:53:21.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.349+0000 7fbc32e2d640 1 -- 172.21.15.133:0/3372564235 wait complete. 2024-02-21T23:53:21.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.350+0000 7fbc32e2d640 1 Processor -- start 2024-02-21T23:53:21.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.350+0000 7fbc32e2d640 1 -- start start 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.351+0000 7fbc32e2d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc2c072f80 0x7fbc2c06b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.351+0000 7fbc32e2d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc2c0b0fe0 0x7fbc2c1efda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.351+0000 7fbc32e2d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c154300 0x7fbc2c1f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.351+0000 7fbc32e2d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1546e0 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.351+0000 7fbc32e2d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c15acf0 con 0x7fbc2c072f80 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.351+0000 7fbc32e2d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0c7640 con 0x7fbc2c154300 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.352+0000 7fbc2bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc2c0b0fe0 0x7fbc2c1efda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.352+0000 7fbc313ca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c154300 0x7fbc2c1f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.352+0000 7fbc30bc9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc2c072f80 0x7fbc2c06b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.352+0000 7fbc2bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc2c0b0fe0 0x7fbc2c1efda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59366/0 (socket says 172.21.15.133:59366) 2024-02-21T23:53:21.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.352+0000 7fbc2bfff640 1 -- 172.21.15.133:0/2884836957 learned_addr learned my addr 172.21.15.133:0/2884836957 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:21.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.356+0000 7fbc2bfff640 1 -- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c154300 msgr2=0x7fbc2c1f02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.356+0000 7fbc2bfff640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c154300 0x7fbc2c1f02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.356+0000 7fbc2bfff640 1 -- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc2c072f80 msgr2=0x7fbc2c06b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.356+0000 7fbc2bfff640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc2c072f80 0x7fbc2c06b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.356+0000 7fbc2bfff640 1 -- 172.21.15.133:0/2884836957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c1f39e0 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.358+0000 7fbc2bfff640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc2c0b0fe0 0x7fbc2c1efda0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbc2002bfd0 tx=0x7fbc200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:21.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.361+0000 7fbc297fa640 1 -- 172.21.15.133:0/2884836957 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20034040 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.361+0000 7fbc297fa640 1 -- 172.21.15.133:0/2884836957 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc20030680 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.361+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c14b1f0 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.361+0000 7fbc297fa640 1 -- 172.21.15.133:0/2884836957 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20006e10 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.361+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c147cb0 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.362+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c06a130 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.606+0000 7fbc297fa640 1 -- 172.21.15.133:0/2884836957 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc20037070 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.606+0000 7fbc297fa640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbc1806b520 0x7fbc1806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.607+0000 7fbc30bc9640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbc1806b520 0x7fbc1806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.607+0000 7fbc30bc9640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbc1806b520 0x7fbc1806d9e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c007fd0 tx=0x7fbc1c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:21.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.619+0000 7fbc297fa640 1 -- 172.21.15.133:0/2884836957 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fbc20030830 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.619+0000 7fbc297fa640 1 -- 172.21.15.133:0/2884836957 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fbc2007b640 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.794+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc2c06c880 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.799+0000 7fbc297fa640 1 -- 172.21.15.133:0/2884836957 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc2007f6e0 con 0x7fbc2c0b0fe0 2024-02-21T23:53:21.793 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-02-21T23:53:21.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbc1806b520 msgr2=0x7fbc1806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbc1806b520 0x7fbc1806d9e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c007fd0 tx=0x7fbc1c009040 comp rx=0 tx=0).stop 2024-02-21T23:53:21.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc2c0b0fe0 msgr2=0x7fbc2c1efda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc2c0b0fe0 0x7fbc2c1efda0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbc2002bfd0 tx=0x7fbc200046c0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 shutdown_connections 2024-02-21T23:53:21.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fbc1806b520 0x7fbc1806d9e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc2c154300 0x7fbc2c1f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc2c0b0fe0 0x7fbc2c1efda0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 --2- 172.21.15.133:0/2884836957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc2c072f80 0x7fbc2c06b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 >> 172.21.15.133:0/2884836957 conn(0x7fbc2c0cc620 msgr2=0x7fbc2c0c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:21.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 shutdown_connections 2024-02-21T23:53:21.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.802+0000 7fbc32e2d640 1 -- 172.21.15.133:0/2884836957 wait complete. 2024-02-21T23:53:21.807 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-21T23:53:21.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.916+0000 7f696b488640 1 Processor -- start 2024-02-21T23:53:21.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.916+0000 7f696b488640 1 -- start start 2024-02-21T23:53:21.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.917+0000 7f696b488640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f69640c9450 0x7f6964072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.917+0000 7f696b488640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f69640c7790 0x7f696415d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.917+0000 7f696b488640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c9830 0x7f696415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.917+0000 7f696b488640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6964072fa0 con 0x7f69640c9450 2024-02-21T23:53:21.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.917+0000 7f696b488640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6964073210 con 0x7f69640c7790 2024-02-21T23:53:21.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.917+0000 7f696b488640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696415a280 con 0x7f69640c9830 2024-02-21T23:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.918+0000 7f6968a23640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f69640c7790 0x7f696415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.918+0000 7f6968a23640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f69640c7790 0x7f696415d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:55830/0 (socket says 172.21.15.133:55830) 2024-02-21T23:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.918+0000 7f6968a23640 1 -- 172.21.15.133:0/1041847525 learned_addr learned my addr 172.21.15.133:0/1041847525 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.918+0000 7f6969a25640 1 --2- 172.21.15.133:0/1041847525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c9830 0x7f696415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.919+0000 7f6969224640 1 --2- 172.21.15.133:0/1041847525 >> v2:172.21.15.133:3300/0 conn(0x7f69640c9450 0x7f6964072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.921+0000 7f6969a25640 1 -- 172.21.15.133:0/1041847525 >> v2:172.21.15.142:3300/0 conn(0x7f69640c7790 msgr2=0x7f696415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.921+0000 7f6969a25640 1 --2- 172.21.15.133:0/1041847525 >> v2:172.21.15.142:3300/0 conn(0x7f69640c7790 0x7f696415d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.921+0000 7f6969a25640 1 -- 172.21.15.133:0/1041847525 >> v2:172.21.15.133:3300/0 conn(0x7f69640c9450 msgr2=0x7f6964072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.921+0000 7f6969a25640 1 --2- 172.21.15.133:0/1041847525 >> v2:172.21.15.133:3300/0 conn(0x7f69640c9450 0x7f6964072a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.921+0000 7f6969a25640 1 -- 172.21.15.133:0/1041847525 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696415a560 con 0x7f69640c9830 2024-02-21T23:53:21.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.925+0000 7f6969a25640 1 --2- 172.21.15.133:0/1041847525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c9830 0x7f696415dca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f696406cee0 tx=0x7f695c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c31c5ec3eb255c server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.929+0000 7f695bfff640 1 -- 172.21.15.133:0/1041847525 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c035020 con 0x7f69640c9830 2024-02-21T23:53:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.929+0000 7f695bfff640 1 -- 172.21.15.133:0/1041847525 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c031830 con 0x7f69640c9830 2024-02-21T23:53:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.929+0000 7f695bfff640 1 -- 172.21.15.133:0/1041847525 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c031c00 con 0x7f69640c9830 2024-02-21T23:53:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.930+0000 7f696b488640 1 -- 172.21.15.133:0/1041847525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c9830 msgr2=0x7f696415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.930+0000 7f696b488640 1 --2- 172.21.15.133:0/1041847525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c9830 0x7f696415dca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f696406cee0 tx=0x7f695c02b8d0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.930+0000 7f696b488640 1 -- 172.21.15.133:0/1041847525 shutdown_connections 2024-02-21T23:53:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.930+0000 7f696b488640 1 --2- 172.21.15.133:0/1041847525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c9830 0x7f696415dca0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.930+0000 7f696b488640 1 --2- 172.21.15.133:0/1041847525 >> v2:172.21.15.142:3300/0 conn(0x7f69640c7790 0x7f696415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.930+0000 7f696b488640 1 --2- 172.21.15.133:0/1041847525 >> v2:172.21.15.133:3300/0 conn(0x7f69640c9450 0x7f6964072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.930+0000 7f696b488640 1 -- 172.21.15.133:0/1041847525 >> 172.21.15.133:0/1041847525 conn(0x7f696406bfd0 msgr2=0x7f696406c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.930+0000 7f696b488640 1 -- 172.21.15.133:0/1041847525 shutdown_connections 2024-02-21T23:53:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.930+0000 7f696b488640 1 -- 172.21.15.133:0/1041847525 wait complete. 2024-02-21T23:53:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.931+0000 7f696b488640 1 Processor -- start 2024-02-21T23:53:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.931+0000 7f696b488640 1 -- start start 2024-02-21T23:53:21.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.932+0000 7f696b488640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c7790 0x7f69640cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.932+0000 7f696b488640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69640c9450 0x7f6964152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.932+0000 7f696b488640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69640c9830 0x7f6964152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.932+0000 7f696b488640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69640709b0 con 0x7f69640c9830 2024-02-21T23:53:21.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.932+0000 7f696b488640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696415ab20 con 0x7f69640c9450 2024-02-21T23:53:21.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.932+0000 7f696b488640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6964070b30 con 0x7f69640c7790 2024-02-21T23:53:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.932+0000 7f6969a25640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69640c9830 0x7f6964152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.933+0000 7f6969224640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c7790 0x7f69640cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.933+0000 7f6968a23640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69640c9450 0x7f6964152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.933+0000 7f6969224640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c7790 0x7f69640cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:49386/0 (socket says 172.21.15.133:49386) 2024-02-21T23:53:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.933+0000 7f6969224640 1 -- 172.21.15.133:0/2255110415 learned_addr learned my addr 172.21.15.133:0/2255110415 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.936+0000 7f6969a25640 1 -- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c7790 msgr2=0x7f69640cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.936+0000 7f6969a25640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c7790 0x7f69640cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.936+0000 7f6969a25640 1 -- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69640c9450 msgr2=0x7f6964152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.936+0000 7f6969a25640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69640c9450 0x7f6964152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.936+0000 7f6969a25640 1 -- 172.21.15.133:0/2255110415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964151930 con 0x7f69640c9830 2024-02-21T23:53:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.936+0000 7f6969224640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c7790 0x7f69640cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:53:21.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.938+0000 7f6969a25640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69640c9830 0x7f6964152ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f695c033000 tx=0x7f695c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:21.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.942+0000 7f6959ffb640 1 -- 172.21.15.133:0/2255110415 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c035040 con 0x7f69640c9830 2024-02-21T23:53:21.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.942+0000 7f6959ffb640 1 -- 172.21.15.133:0/2255110415 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c03b640 con 0x7f69640c9830 2024-02-21T23:53:21.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.942+0000 7f6959ffb640 1 -- 172.21.15.133:0/2255110415 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c03ba30 con 0x7f69640c9830 2024-02-21T23:53:21.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.942+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6964151ba0 con 0x7f69640c9830 2024-02-21T23:53:21.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.942+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6964152160 con 0x7f69640c9830 2024-02-21T23:53:21.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.943+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696406a130 con 0x7f69640c9830 2024-02-21T23:53:21.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.962+0000 7f6959ffb640 1 -- 172.21.15.133:0/2255110415 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f695c038020 con 0x7f69640c9830 2024-02-21T23:53:21.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.962+0000 7f6959ffb640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f694806b4d0 0x7f694806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:21.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.963+0000 7f6969224640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f694806b4d0 0x7f694806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.963+0000 7f6969224640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f694806b4d0 0x7f694806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f69600094c0 tx=0x7f696000b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:21.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:21.966+0000 7f6959ffb640 1 -- 172.21.15.133:0/2255110415 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f695c078f80 con 0x7f69640c9830 2024-02-21T23:53:22.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.203+0000 7f6959ffb640 1 -- 172.21.15.133:0/2255110415 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f695c054020 con 0x7f69640c9830 2024-02-21T23:53:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.420+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f696406c530 con 0x7f69640c9830 2024-02-21T23:53:22.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.427+0000 7f6959ffb640 1 -- 172.21.15.133:0/2255110415 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f695c07aee0 con 0x7f69640c9830 2024-02-21T23:53:22.421 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:53:22.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f694806b4d0 msgr2=0x7f694806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:22.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f694806b4d0 0x7f694806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f69600094c0 tx=0x7f696000b040 comp rx=0 tx=0).stop 2024-02-21T23:53:22.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69640c9830 msgr2=0x7f6964152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:22.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69640c9830 0x7f6964152ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f695c033000 tx=0x7f695c0046c0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 shutdown_connections 2024-02-21T23:53:22.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f694806b4d0 0x7f694806d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69640c9830 0x7f6964152ea0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69640c9450 0x7f6964152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 --2- 172.21.15.133:0/2255110415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69640c7790 0x7f69640cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 >> 172.21.15.133:0/2255110415 conn(0x7f696406bfd0 msgr2=0x7f69640cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:22.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 shutdown_connections 2024-02-21T23:53:22.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.430+0000 7f696b488640 1 -- 172.21.15.133:0/2255110415 wait complete. 2024-02-21T23:53:22.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-21T23:53:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.545+0000 7fb1acdcc640 1 Processor -- start 2024-02-21T23:53:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.545+0000 7fb1acdcc640 1 -- start start 2024-02-21T23:53:22.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.546+0000 7fb1acdcc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb1a8150a30 0x7fb1a80c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:22.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.546+0000 7fb1acdcc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a80c7ac0 0x7fb1a806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:22.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.546+0000 7fb1acdcc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a80c53e0 con 0x7fb1a8071f60 2024-02-21T23:53:22.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.546+0000 7fb1acdcc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a8071860 con 0x7fb1a8150a30 2024-02-21T23:53:22.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.546+0000 7fb1acdcc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8072540 con 0x7fb1a80c7ac0 2024-02-21T23:53:22.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.547+0000 7fb1a659c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb1a8150a30 0x7fb1a80c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:22.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.547+0000 7fb1a6d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a80c7ac0 0x7fb1a806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:22.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.547+0000 7fb1a5d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb1a8071f60 0x7fb1a8071150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44430/0 (socket says 172.21.15.133:44430) 2024-02-21T23:53:22.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.547+0000 7fb1a5d9b640 1 -- 172.21.15.133:0/2014905085 learned_addr learned my addr 172.21.15.133:0/2014905085 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.549+0000 7fb1a559a640 1 -- 172.21.15.133:0/2014905085 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595364838 0 0) 0x7fb1a80c53e0 con 0x7fb1a8071f60 2024-02-21T23:53:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.549+0000 7fb1a559a640 1 -- 172.21.15.133:0/2014905085 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb190003520 con 0x7fb1a8071f60 2024-02-21T23:53:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.549+0000 7fb1a6d9d640 1 -- 172.21.15.133:0/2014905085 >> v2:172.21.15.142:3300/0 conn(0x7fb1a8150a30 msgr2=0x7fb1a80c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.549+0000 7fb1a6d9d640 1 --2- 172.21.15.133:0/2014905085 >> v2:172.21.15.142:3300/0 conn(0x7fb1a8150a30 0x7fb1a80c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.549+0000 7fb1a6d9d640 1 -- 172.21.15.133:0/2014905085 >> v1:172.21.15.133:6789/0 conn(0x7fb1a8071f60 legacy=0x7fb1a8071150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.549+0000 7fb1a6d9d640 1 -- 172.21.15.133:0/2014905085 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a80c5680 con 0x7fb1a80c7ac0 2024-02-21T23:53:22.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.551+0000 7fb1a6d9d640 1 --2- 172.21.15.133:0/2014905085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a80c7ac0 0x7fb1a806e8b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80726c0 tx=0x7fb1a002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=706465c25d19ad41 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:22.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.555+0000 7fb1a559a640 1 -- 172.21.15.133:0/2014905085 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0035020 con 0x7fb1a80c7ac0 2024-02-21T23:53:22.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.555+0000 7fb1a559a640 1 -- 172.21.15.133:0/2014905085 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a0031890 con 0x7fb1a80c7ac0 2024-02-21T23:53:22.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.555+0000 7fb1a559a640 1 -- 172.21.15.133:0/2014905085 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0031c60 con 0x7fb1a80c7ac0 2024-02-21T23:53:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.556+0000 7fb1acdcc640 1 -- 172.21.15.133:0/2014905085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a80c7ac0 msgr2=0x7fb1a806e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.556+0000 7fb1acdcc640 1 --2- 172.21.15.133:0/2014905085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a80c7ac0 0x7fb1a806e8b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80726c0 tx=0x7fb1a002b930 comp rx=0 tx=0).stop 2024-02-21T23:53:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.556+0000 7fb1acdcc640 1 -- 172.21.15.133:0/2014905085 shutdown_connections 2024-02-21T23:53:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.556+0000 7fb1acdcc640 1 --2- 172.21.15.133:0/2014905085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a80c7ac0 0x7fb1a806e8b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.556+0000 7fb1acdcc640 1 --2- 172.21.15.133:0/2014905085 >> v2:172.21.15.142:3300/0 conn(0x7fb1a8150a30 0x7fb1a80c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.556+0000 7fb1acdcc640 1 -- 172.21.15.133:0/2014905085 >> 172.21.15.133:0/2014905085 conn(0x7fb1a806aec0 msgr2=0x7fb1a806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.556+0000 7fb1acdcc640 1 -- 172.21.15.133:0/2014905085 shutdown_connections 2024-02-21T23:53:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.556+0000 7fb1acdcc640 1 -- 172.21.15.133:0/2014905085 wait complete. 2024-02-21T23:53:22.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.557+0000 7fb1acdcc640 1 Processor -- start 2024-02-21T23:53:22.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.557+0000 7fb1acdcc640 1 -- start start 2024-02-21T23:53:22.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.558+0000 7fb1acdcc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a8071f60 0x7fb1a80c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:22.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.558+0000 7fb1acdcc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a80c7ac0 0x7fb1a8142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:22.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.558+0000 7fb1acdcc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a8150a30 0x7fb1a8151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:22.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.558+0000 7fb1acdcc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8072a70 con 0x7fb1a8150a30 2024-02-21T23:53:22.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.558+0000 7fb1acdcc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8072790 con 0x7fb1a80c7ac0 2024-02-21T23:53:22.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.558+0000 7fb1acdcc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8150e10 con 0x7fb1a8071f60 2024-02-21T23:53:22.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.558+0000 7fb1a659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a8071f60 0x7fb1a80c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:22.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.558+0000 7fb1a6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a8150a30 0x7fb1a8151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:22.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.559+0000 7fb1a5d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a80c7ac0 0x7fb1a8142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:22.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.559+0000 7fb1a659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a8071f60 0x7fb1a80c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:49392/0 (socket says 172.21.15.133:49392) 2024-02-21T23:53:22.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.559+0000 7fb1a659c640 1 -- 172.21.15.133:0/3553223347 learned_addr learned my addr 172.21.15.133:0/3553223347 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:22.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.562+0000 7fb1a6d9d640 1 -- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a8071f60 msgr2=0x7fb1a80c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:22.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.562+0000 7fb1a6d9d640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a8071f60 0x7fb1a80c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.562+0000 7fb1a6d9d640 1 -- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a80c7ac0 msgr2=0x7fb1a8142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:22.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.562+0000 7fb1a6d9d640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a80c7ac0 0x7fb1a8142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:22.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.562+0000 7fb1a6d9d640 1 -- 172.21.15.133:0/3553223347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a80ca360 con 0x7fb1a8150a30 2024-02-21T23:53:22.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.562+0000 7fb1a659c640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a8071f60 0x7fb1a80c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:53:22.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.564+0000 7fb1a6d9d640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a8150a30 0x7fb1a8151600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0033000 tx=0x7fb1a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:22.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.568+0000 7fb196ffd640 1 -- 172.21.15.133:0/3553223347 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0035040 con 0x7fb1a8150a30 2024-02-21T23:53:22.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.568+0000 7fb196ffd640 1 -- 172.21.15.133:0/3553223347 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a0030ca0 con 0x7fb1a8150a30 2024-02-21T23:53:22.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.568+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a80ca640 con 0x7fb1a8150a30 2024-02-21T23:53:22.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.568+0000 7fb196ffd640 1 -- 172.21.15.133:0/3553223347 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0006e10 con 0x7fb1a8150a30 2024-02-21T23:53:22.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.568+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a80cac00 con 0x7fb1a8150a30 2024-02-21T23:53:22.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.569+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a8072f90 con 0x7fb1a8150a30 2024-02-21T23:53:22.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.586+0000 7fb196ffd640 1 -- 172.21.15.133:0/3553223347 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1a0038020 con 0x7fb1a8150a30 2024-02-21T23:53:22.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.587+0000 7fb196ffd640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb18806b4d0 0x7fb18806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:22.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.587+0000 7fb1a659c640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb18806b4d0 0x7fb18806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:22.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.587+0000 7fb1a659c640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb18806b4d0 0x7fb18806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1980094c0 tx=0x7fb19800b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:22.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.826+0000 7fb196ffd640 1 -- 172.21.15.133:0/3553223347 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fb1a0079420 con 0x7fb1a8150a30 2024-02-21T23:53:22.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:22.826+0000 7fb196ffd640 1 -- 172.21.15.133:0/3553223347 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fb1a8072f90 con 0x7fb1a8150a30 2024-02-21T23:53:23.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.015+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1a806a130 con 0x7fb1a8150a30 2024-02-21T23:53:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.019+0000 7fb196ffd640 1 -- 172.21.15.133:0/3553223347 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1a002d040 con 0x7fb1a8150a30 2024-02-21T23:53:23.013 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb18806b4d0 msgr2=0x7fb18806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb18806b4d0 0x7fb18806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1980094c0 tx=0x7fb19800b040 comp rx=0 tx=0).stop 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a8150a30 msgr2=0x7fb1a8151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a8150a30 0x7fb1a8151600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0033000 tx=0x7fb1a00046c0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 shutdown_connections 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb18806b4d0 0x7fb18806d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a8150a30 0x7fb1a8151600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a80c7ac0 0x7fb1a8142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 --2- 172.21.15.133:0/3553223347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a8071f60 0x7fb1a80c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.021+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 >> 172.21.15.133:0/3553223347 conn(0x7fb1a806aec0 msgr2=0x7fb1a8071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:23.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.022+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 shutdown_connections 2024-02-21T23:53:23.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.022+0000 7fb1acdcc640 1 -- 172.21.15.133:0/3553223347 wait complete. 2024-02-21T23:53:23.027 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-21T23:53:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.136+0000 7f7ec3c36640 1 Processor -- start 2024-02-21T23:53:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.136+0000 7f7ec3c36640 1 -- start start 2024-02-21T23:53:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.137+0000 7f7ec3c36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc154300 0x7f7ebc06d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.137+0000 7f7ec3c36640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7ebc0b0fe0 0x7f7ebc072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.137+0000 7f7ec3c36640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc15afd0 con 0x7f7ebc072f80 2024-02-21T23:53:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.137+0000 7f7ec3c36640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ebc15a170 con 0x7f7ebc0b0fe0 2024-02-21T23:53:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.137+0000 7f7ec3c36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc15a450 con 0x7f7ebc154300 2024-02-21T23:53:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.138+0000 7f7ec11d1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7ebc0b0fe0 0x7f7ebc072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.138+0000 7f7ec19d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc154300 0x7f7ebc06d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.138+0000 7f7ec19d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc154300 0x7f7ebc06d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:49402/0 (socket says 172.21.15.133:49402) 2024-02-21T23:53:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.138+0000 7f7ec19d2640 1 -- 172.21.15.133:0/2230277009 learned_addr learned my addr 172.21.15.133:0/2230277009 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:23.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.140+0000 7f7ec19d2640 1 -- 172.21.15.133:0/2230277009 >> v2:172.21.15.133:3300/0 conn(0x7f7ebc0b0fe0 msgr2=0x7f7ebc072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:23.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.140+0000 7f7ec19d2640 1 --2- 172.21.15.133:0/2230277009 >> v2:172.21.15.133:3300/0 conn(0x7f7ebc0b0fe0 0x7f7ebc072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.140+0000 7f7ec19d2640 1 -- 172.21.15.133:0/2230277009 >> v1:172.21.15.133:6789/0 conn(0x7f7ebc072f80 legacy=0x7f7ebc06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.140+0000 7f7ec19d2640 1 -- 172.21.15.133:0/2230277009 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc15a730 con 0x7f7ebc154300 2024-02-21T23:53:23.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.142+0000 7f7ec19d2640 1 --2- 172.21.15.133:0/2230277009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc154300 0x7f7ebc06d0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009e50 tx=0x7f7eb802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b30824e0eb5eedd server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:23.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.144+0000 7f7ec09d0640 1 -- 172.21.15.133:0/2230277009 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb802cb90 con 0x7f7ebc154300 2024-02-21T23:53:23.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec09d0640 1 -- 172.21.15.133:0/2230277009 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb802cd30 con 0x7f7ebc154300 2024-02-21T23:53:23.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec09d0640 1 -- 172.21.15.133:0/2230277009 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8034610 con 0x7f7ebc154300 2024-02-21T23:53:23.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec3c36640 1 -- 172.21.15.133:0/2230277009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc154300 msgr2=0x7f7ebc06d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec3c36640 1 --2- 172.21.15.133:0/2230277009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc154300 0x7f7ebc06d0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009e50 tx=0x7f7eb802bff0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec3c36640 1 -- 172.21.15.133:0/2230277009 shutdown_connections 2024-02-21T23:53:23.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec3c36640 1 --2- 172.21.15.133:0/2230277009 >> v2:172.21.15.133:3300/0 conn(0x7f7ebc0b0fe0 0x7f7ebc072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec3c36640 1 --2- 172.21.15.133:0/2230277009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc154300 0x7f7ebc06d0a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec3c36640 1 -- 172.21.15.133:0/2230277009 >> 172.21.15.133:0/2230277009 conn(0x7f7ebc0cc620 msgr2=0x7f7ebc0c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:23.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec3c36640 1 -- 172.21.15.133:0/2230277009 shutdown_connections 2024-02-21T23:53:23.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.146+0000 7f7ec3c36640 1 -- 172.21.15.133:0/2230277009 wait complete. 2024-02-21T23:53:23.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.148+0000 7f7ec3c36640 1 Processor -- start 2024-02-21T23:53:23.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.148+0000 7f7ec3c36640 1 -- start start 2024-02-21T23:53:23.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.148+0000 7f7ec3c36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc072f80 0x7f7ebc06e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.148+0000 7f7ec3c36640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ebc0b0fe0 0x7f7ebc14ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.148+0000 7f7ec3c36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ebc154300 0x7f7ebc14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.148+0000 7f7ec3c36640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc1546e0 con 0x7f7ebc154300 2024-02-21T23:53:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.148+0000 7f7ec3c36640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc15acf0 con 0x7f7ebc0b0fe0 2024-02-21T23:53:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.148+0000 7f7ec3c36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc06d5e0 con 0x7f7ebc072f80 2024-02-21T23:53:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.149+0000 7f7ec19d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc072f80 0x7f7ebc06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.149+0000 7f7ec21d3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ebc154300 0x7f7ebc14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.149+0000 7f7ec19d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc072f80 0x7f7ebc06e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:49408/0 (socket says 172.21.15.133:49408) 2024-02-21T23:53:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.149+0000 7f7ec19d2640 1 -- 172.21.15.133:0/1461110865 learned_addr learned my addr 172.21.15.133:0/1461110865 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.149+0000 7f7ec11d1640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ebc0b0fe0 0x7f7ebc14ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.152+0000 7f7ec21d3640 1 -- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc072f80 msgr2=0x7f7ebc06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.152+0000 7f7ec21d3640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc072f80 0x7f7ebc06e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.152+0000 7f7ec21d3640 1 -- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ebc0b0fe0 msgr2=0x7f7ebc14ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.152+0000 7f7ec21d3640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ebc0b0fe0 0x7f7ebc14ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.152+0000 7f7ec21d3640 1 -- 172.21.15.133:0/1461110865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc1efc20 con 0x7f7ebc154300 2024-02-21T23:53:23.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.152+0000 7f7ec19d2640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc072f80 0x7f7ebc06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:53:23.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.154+0000 7f7ec21d3640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ebc154300 0x7f7ebc14d380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400c3c0 tx=0x7f7eb400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.158+0000 7f7eb27fc640 1 -- 172.21.15.133:0/1461110865 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb40052d0 con 0x7f7ebc154300 2024-02-21T23:53:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.158+0000 7f7eb27fc640 1 -- 172.21.15.133:0/1461110865 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb4005470 con 0x7f7ebc154300 2024-02-21T23:53:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.158+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc1eff00 con 0x7f7ebc154300 2024-02-21T23:53:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.158+0000 7f7eb27fc640 1 -- 172.21.15.133:0/1461110865 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4007260 con 0x7f7ebc154300 2024-02-21T23:53:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.158+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc1f04c0 con 0x7f7ebc154300 2024-02-21T23:53:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.159+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc06a130 con 0x7f7ebc154300 2024-02-21T23:53:23.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.403+0000 7f7eb27fc640 1 -- 172.21.15.133:0/1461110865 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7eb4007400 con 0x7f7ebc154300 2024-02-21T23:53:23.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.404+0000 7f7eb27fc640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f7ea006b520 0x7f7ea006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.404+0000 7f7ec19d2640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f7ea006b520 0x7f7ea006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.405+0000 7f7ec19d2640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f7ea006b520 0x7f7ea006d9e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000f80 tx=0x7f7eb8030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.416+0000 7f7eb27fc640 1 -- 172.21.15.133:0/1461110865 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f7eb4056630 con 0x7f7ebc154300 2024-02-21T23:53:23.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.416+0000 7f7eb27fc640 1 -- 172.21.15.133:0/1461110865 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f7eb4058590 con 0x7f7ebc154300 2024-02-21T23:53:23.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.601+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7ebc06c880 con 0x7f7ebc154300 2024-02-21T23:53:23.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.606+0000 7f7eb27fc640 1 -- 172.21.15.133:0/1461110865 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7eb405c630 con 0x7f7ebc154300 2024-02-21T23:53:23.600 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-02-21T23:53:23.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f7ea006b520 msgr2=0x7f7ea006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f7ea006b520 0x7f7ea006d9e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000f80 tx=0x7f7eb8030000 comp rx=0 tx=0).stop 2024-02-21T23:53:23.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ebc154300 msgr2=0x7f7ebc14d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ebc154300 0x7f7ebc14d380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400c3c0 tx=0x7f7eb400c880 comp rx=0 tx=0).stop 2024-02-21T23:53:23.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 shutdown_connections 2024-02-21T23:53:23.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f7ea006b520 0x7f7ea006d9e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ebc154300 0x7f7ebc14d380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ebc0b0fe0 0x7f7ebc14ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 --2- 172.21.15.133:0/1461110865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ebc072f80 0x7f7ebc06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 >> 172.21.15.133:0/1461110865 conn(0x7f7ebc0cc620 msgr2=0x7f7ebc0c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:23.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 shutdown_connections 2024-02-21T23:53:23.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.608+0000 7f7ec3c36640 1 -- 172.21.15.133:0/1461110865 wait complete. 2024-02-21T23:53:23.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-21T23:53:23.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.722+0000 7f661cd08640 1 Processor -- start 2024-02-21T23:53:23.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.722+0000 7f661cd08640 1 -- start start 2024-02-21T23:53:23.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.722+0000 7f661cd08640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 0x7f66180cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.722+0000 7f661cd08640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661815ab40 con 0x7f66180c6f50 2024-02-21T23:53:23.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.722+0000 7f661cd08640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661815ae00 con 0x7f6618154600 2024-02-21T23:53:23.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.722+0000 7f661cd08640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180c74a0 con 0x7f66180c8320 2024-02-21T23:53:23.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.723+0000 7f661659c640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f6618154600 0x7f66180ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:37888/0 (socket says 172.21.15.133:37888) 2024-02-21T23:53:23.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.723+0000 7f661659c640 1 -- 172.21.15.133:0/1664316865 learned_addr learned my addr 172.21.15.133:0/1664316865 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:23.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.724+0000 7f6616d9d640 1 --2- 172.21.15.133:0/1664316865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 0x7f66180cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.725+0000 7f6615d9b640 1 -- 172.21.15.133:0/1664316865 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673199687 0 0) 0x7f661815ab40 con 0x7f66180c6f50 2024-02-21T23:53:23.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.725+0000 7f6615d9b640 1 -- 172.21.15.133:0/1664316865 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6604003520 con 0x7f66180c6f50 2024-02-21T23:53:23.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.725+0000 7f6615d9b640 1 -- 172.21.15.133:0/1664316865 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549800449 0 0) 0x7f661815ae00 con 0x7f6618154600 2024-02-21T23:53:23.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.726+0000 7f6615d9b640 1 -- 172.21.15.133:0/1664316865 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f661815ab40 con 0x7f6618154600 2024-02-21T23:53:23.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.729+0000 7f6616d9d640 1 -- 172.21.15.133:0/1664316865 >> v1:172.21.15.142:6789/0 conn(0x7f6618154600 legacy=0x7f66180ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.729+0000 7f6616d9d640 1 -- 172.21.15.133:0/1664316865 >> v1:172.21.15.133:6789/0 conn(0x7f66180c6f50 legacy=0x7f66180c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.729+0000 7f6615d9b640 1 -- 172.21.15.133:0/1664316865 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3003371706 0 0) 0x7f6604003520 con 0x7f66180c6f50 2024-02-21T23:53:23.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.729+0000 7f6616d9d640 1 -- 172.21.15.133:0/1664316865 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180cb580 con 0x7f66180c8320 2024-02-21T23:53:23.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.731+0000 7f6616d9d640 1 --2- 172.21.15.133:0/1664316865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 0x7f66180cbc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66180c7620 tx=0x7f661002bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81c44b3ef576632 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:23.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.735+0000 7f6615d9b640 1 -- 172.21.15.133:0/1664316865 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610039070 con 0x7f66180c8320 2024-02-21T23:53:23.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.735+0000 7f6615d9b640 1 -- 172.21.15.133:0/1664316865 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661002c910 con 0x7f66180c8320 2024-02-21T23:53:23.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.735+0000 7f6615d9b640 1 -- 172.21.15.133:0/1664316865 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600001670 con 0x7f66180c8320 2024-02-21T23:53:23.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.736+0000 7f661cd08640 1 -- 172.21.15.133:0/1664316865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 msgr2=0x7f66180cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.736+0000 7f661cd08640 1 --2- 172.21.15.133:0/1664316865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 0x7f66180cbc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66180c7620 tx=0x7f661002bde0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.736+0000 7f661cd08640 1 -- 172.21.15.133:0/1664316865 shutdown_connections 2024-02-21T23:53:23.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.736+0000 7f661cd08640 1 --2- 172.21.15.133:0/1664316865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 0x7f66180cbc00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.736+0000 7f661cd08640 1 -- 172.21.15.133:0/1664316865 >> 172.21.15.133:0/1664316865 conn(0x7f6618062000 msgr2=0x7f66180cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:23.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.736+0000 7f661cd08640 1 -- 172.21.15.133:0/1664316865 shutdown_connections 2024-02-21T23:53:23.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.736+0000 7f661cd08640 1 -- 172.21.15.133:0/1664316865 wait complete. 2024-02-21T23:53:23.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.738+0000 7f661cd08640 1 Processor -- start 2024-02-21T23:53:23.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.738+0000 7f661cd08640 1 -- start start 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.738+0000 7f661cd08640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66180c6f50 0x7f66180c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.738+0000 7f661cd08640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 0x7f661814bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.738+0000 7f661cd08640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6618154600 0x7f66181efd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.738+0000 7f661cd08640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66180b5720 con 0x7f6618154600 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.738+0000 7f661cd08640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66180c7780 con 0x7f66180c6f50 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.738+0000 7f661cd08640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180b58a0 con 0x7f66180c8320 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.739+0000 7f661659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66180c6f50 0x7f66180c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.739+0000 7f6616d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6618154600 0x7f66181efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.739+0000 7f660ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 0x7f661814bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.739+0000 7f661659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66180c6f50 0x7f66180c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:55896/0 (socket says 172.21.15.133:55896) 2024-02-21T23:53:23.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.739+0000 7f661659c640 1 -- 172.21.15.133:0/981183985 learned_addr learned my addr 172.21.15.133:0/981183985 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:23.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.742+0000 7f6616d9d640 1 -- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 msgr2=0x7f661814bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.742+0000 7f6616d9d640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 0x7f661814bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.742+0000 7f6616d9d640 1 -- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66180c6f50 msgr2=0x7f66180c33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:23.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.742+0000 7f6616d9d640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66180c6f50 0x7f66180c33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:23.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.742+0000 7f6616d9d640 1 -- 172.21.15.133:0/981183985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618156420 con 0x7f6618154600 2024-02-21T23:53:23.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.745+0000 7f6616d9d640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6618154600 0x7f66181efd00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6610033000 tx=0x7f6610004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:23.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.749+0000 7f660effd640 1 -- 172.21.15.133:0/981183985 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610039050 con 0x7f6618154600 2024-02-21T23:53:23.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.749+0000 7f660effd640 1 -- 172.21.15.133:0/981183985 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661003a040 con 0x7f6618154600 2024-02-21T23:53:23.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.749+0000 7f660effd640 1 -- 172.21.15.133:0/981183985 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610009450 con 0x7f6618154600 2024-02-21T23:53:23.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.749+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6618156690 con 0x7f6618154600 2024-02-21T23:53:23.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.749+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66181f4000 con 0x7f6618154600 2024-02-21T23:53:23.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.750+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661806a130 con 0x7f6618154600 2024-02-21T23:53:23.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.767+0000 7f660effd640 1 -- 172.21.15.133:0/981183985 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6610047020 con 0x7f6618154600 2024-02-21T23:53:23.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.768+0000 7f660effd640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f65f806b4d0 0x7f65f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:23.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.768+0000 7f661659c640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f65f806b4d0 0x7f65f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:23.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:23.768+0000 7f661659c640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f65f806b4d0 0x7f65f806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6608000f60 tx=0x7f6608009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:23.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.004+0000 7f660effd640 1 -- 172.21.15.133:0/981183985 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6610034dd0 con 0x7f6618154600 2024-02-21T23:53:23.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.004+0000 7f660effd640 1 -- 172.21.15.133:0/981183985 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f661007b390 con 0x7f6618154600 2024-02-21T23:53:24.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.189+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6618159e90 con 0x7f6618154600 2024-02-21T23:53:24.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.194+0000 7f660effd640 1 -- 172.21.15.133:0/981183985 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f661007f430 con 0x7f6618154600 2024-02-21T23:53:24.188 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-02-21T23:53:24.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f65f806b4d0 msgr2=0x7f65f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:24.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f65f806b4d0 0x7f65f806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6608000f60 tx=0x7f6608009040 comp rx=0 tx=0).stop 2024-02-21T23:53:24.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6618154600 msgr2=0x7f66181efd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:24.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6618154600 0x7f66181efd00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6610033000 tx=0x7f6610004a40 comp rx=0 tx=0).stop 2024-02-21T23:53:24.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 shutdown_connections 2024-02-21T23:53:24.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f65f806b4d0 0x7f65f806d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:24.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6618154600 0x7f66181efd00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:24.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66180c8320 0x7f661814bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:24.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 --2- 172.21.15.133:0/981183985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66180c6f50 0x7f66180c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:24.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 >> 172.21.15.133:0/981183985 conn(0x7f6618062000 msgr2=0x7f661814dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:24.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 shutdown_connections 2024-02-21T23:53:24.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:24.196+0000 7f661cd08640 1 -- 172.21.15.133:0/981183985 wait complete. 2024-02-21T23:53:24.201 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-02-21T23:53:24.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:53:24.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-02-21T23:53:24.229 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:53:24.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[46098]: fuse finished with error 0 and tester_r 0 2024-02-21T23:53:24.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:==00:00:27:13.886 46098== 2024-02-21T23:53:24.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:==00:00:27:13.886 46098== Exit program on first error (--exit-on-first-error=yes) 2024-02-21T23:53:24.855 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:daemon-helper: command failed with exit status 42 2024-02-21T23:53:30.331 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-02-21T23:53:30.331 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-02-21T23:53:30.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:53:30.332 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-21T23:53:30.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:53:30.351 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-02-21T23:53:30.408 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-21T23:53:30.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:53:30.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-21T23:53:30.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-21T23:53:30.480 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-02-21T23:53:30.579 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-02-21T23:53:30.590 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-02-21T23:53:30.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-21T23:53:30.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.701+0000 7fb8e942d640 1 Processor -- start 2024-02-21T23:53:30.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.701+0000 7fb8e942d640 1 -- start start 2024-02-21T23:53:30.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.702+0000 7fb8e942d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8e4154600 0x7fb8e40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:30.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.702+0000 7fb8e942d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb8e40c6f50 0x7fb8e40ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:30.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.702+0000 7fb8e942d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e415ab40 con 0x7fb8e40cab80 2024-02-21T23:53:30.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.702+0000 7fb8e942d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e40c74a0 con 0x7fb8e4154600 2024-02-21T23:53:30.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.702+0000 7fb8e942d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e40c7780 con 0x7fb8e40c6f50 2024-02-21T23:53:30.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.703+0000 7fb8dbfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb8e40c6f50 0x7fb8e40ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:30.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.703+0000 7fb8e37fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fb8e40cab80 0x7fb8e40c7c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:37900/0 (socket says 172.21.15.133:37900) 2024-02-21T23:53:30.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.703+0000 7fb8e37fe640 1 -- 172.21.15.133:0/2501608058 learned_addr learned my addr 172.21.15.133:0/2501608058 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:30.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.706+0000 7fb8dbfff640 1 -- 172.21.15.133:0/2501608058 >> v2:172.21.15.133:3300/0 conn(0x7fb8e4154600 msgr2=0x7fb8e40cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:30.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.706+0000 7fb8dbfff640 1 --2- 172.21.15.133:0/2501608058 >> v2:172.21.15.133:3300/0 conn(0x7fb8e4154600 0x7fb8e40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:30.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.706+0000 7fb8dbfff640 1 -- 172.21.15.133:0/2501608058 >> v1:172.21.15.142:6789/0 conn(0x7fb8e40cab80 legacy=0x7fb8e40c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:30.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.706+0000 7fb8dbfff640 1 -- 172.21.15.133:0/2501608058 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e415ae00 con 0x7fb8e40c6f50 2024-02-21T23:53:30.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.710+0000 7fb8dbfff640 1 --2- 172.21.15.133:0/2501608058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 0x7fb8e40ca640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb8e406d390 tx=0x7fb8cc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19755d0e6a8c30d server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:30.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.714+0000 7fb8e27fc640 1 -- 172.21.15.133:0/2501608058 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc035020 con 0x7fb8e40c6f50 2024-02-21T23:53:30.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.714+0000 7fb8e27fc640 1 -- 172.21.15.133:0/2501608058 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc031830 con 0x7fb8e40c6f50 2024-02-21T23:53:30.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.714+0000 7fb8e27fc640 1 -- 172.21.15.133:0/2501608058 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc031c00 con 0x7fb8e40c6f50 2024-02-21T23:53:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.715+0000 7fb8e942d640 1 -- 172.21.15.133:0/2501608058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 msgr2=0x7fb8e40ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.715+0000 7fb8e942d640 1 --2- 172.21.15.133:0/2501608058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 0x7fb8e40ca640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb8e406d390 tx=0x7fb8cc02b8d0 comp rx=0 tx=0).stop 2024-02-21T23:53:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.715+0000 7fb8e942d640 1 -- 172.21.15.133:0/2501608058 shutdown_connections 2024-02-21T23:53:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.715+0000 7fb8e942d640 1 --2- 172.21.15.133:0/2501608058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 0x7fb8e40ca640 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.715+0000 7fb8e942d640 1 --2- 172.21.15.133:0/2501608058 >> v2:172.21.15.133:3300/0 conn(0x7fb8e4154600 0x7fb8e40cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.715+0000 7fb8e942d640 1 -- 172.21.15.133:0/2501608058 >> 172.21.15.133:0/2501608058 conn(0x7fb8e4062000 msgr2=0x7fb8e40cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.715+0000 7fb8e942d640 1 -- 172.21.15.133:0/2501608058 shutdown_connections 2024-02-21T23:53:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.715+0000 7fb8e942d640 1 -- 172.21.15.133:0/2501608058 wait complete. 2024-02-21T23:53:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.716+0000 7fb8e942d640 1 Processor -- start 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.716+0000 7fb8e942d640 1 -- start start 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e942d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 0x7fb8e4156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e942d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cab80 0x7fb8e4156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e942d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e4154600 0x7fb8e41f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e942d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40b5720 con 0x7fb8e4154600 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e942d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40cb860 con 0x7fb8e40c6f50 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e942d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40b58a0 con 0x7fb8e40cab80 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e4154600 0x7fb8e41f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 0x7fb8e4156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 0x7fb8e4156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:55900/0 (socket says 172.21.15.133:55900) 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.717+0000 7fb8e2ffd640 1 -- 172.21.15.133:0/1296043977 learned_addr learned my addr 172.21.15.133:0/1296043977 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.718+0000 7fb8dbfff640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cab80 0x7fb8e4156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:30.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.721+0000 7fb8e2ffd640 1 -- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cab80 msgr2=0x7fb8e4156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:30.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.721+0000 7fb8e2ffd640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cab80 0x7fb8e4156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:30.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.721+0000 7fb8e2ffd640 1 -- 172.21.15.133:0/1296043977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e4154600 msgr2=0x7fb8e41f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:30.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.721+0000 7fb8e2ffd640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e4154600 0x7fb8e41f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:30.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.721+0000 7fb8e2ffd640 1 -- 172.21.15.133:0/1296043977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e4157750 con 0x7fb8e40c6f50 2024-02-21T23:53:30.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.721+0000 7fb8e37fe640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e4154600 0x7fb8e41f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:53:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.723+0000 7fb8e2ffd640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 0x7fb8e4156320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400b410 tx=0x7fb8d400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.727+0000 7fb8db7fe640 1 -- 172.21.15.133:0/1296043977 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4015020 con 0x7fb8e40c6f50 2024-02-21T23:53:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.727+0000 7fb8db7fe640 1 -- 172.21.15.133:0/1296043977 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d4011850 con 0x7fb8e40c6f50 2024-02-21T23:53:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.727+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e4148190 con 0x7fb8e40c6f50 2024-02-21T23:53:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.727+0000 7fb8db7fe640 1 -- 172.21.15.133:0/1296043977 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4011c40 con 0x7fb8e40c6f50 2024-02-21T23:53:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.727+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e415a710 con 0x7fb8e40c6f50 2024-02-21T23:53:30.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.728+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e4068ad0 con 0x7fb8e40c6f50 2024-02-21T23:53:30.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.790+0000 7fb8db7fe640 1 -- 172.21.15.133:0/1296043977 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8d4018020 con 0x7fb8e40c6f50 2024-02-21T23:53:30.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.790+0000 7fb8db7fe640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb8c406b550 0x7fb8c406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:30.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.791+0000 7fb8dbfff640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb8c406b550 0x7fb8c406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:30.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.791+0000 7fb8dbfff640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb8c406b550 0x7fb8c406da10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc036000 tx=0x7fb8cc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.985+0000 7fb8db7fe640 1 -- 172.21.15.133:0/1296043977 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fb8d4011de0 con 0x7fb8e40c6f50 2024-02-21T23:53:30.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:30.996+0000 7fb8db7fe640 1 -- 172.21.15.133:0/1296043977 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fb8e4068ad0 con 0x7fb8e40c6f50 2024-02-21T23:53:31.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.174+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8e40cd160 con 0x7fb8c406b550 2024-02-21T23:53:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.183+0000 7fb8db7fe640 1 -- 172.21.15.133:0/1296043977 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+333996 (secure 0 0 0) 0x7fb8cc09b8c0 con 0x7fb8c406b550 2024-02-21T23:53:31.182 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:53:31.184 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-02-21T23:53:31.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb8c406b550 msgr2=0x7fb8c406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb8c406b550 0x7fb8c406da10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc036000 tx=0x7fb8cc008030 comp rx=0 tx=0).stop 2024-02-21T23:53:31.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 msgr2=0x7fb8e4156320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 0x7fb8e4156320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400b410 tx=0x7fb8d400b8d0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 shutdown_connections 2024-02-21T23:53:31.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb8c406b550 0x7fb8c406da10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e4154600 0x7fb8e41f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cab80 0x7fb8e4156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 --2- 172.21.15.133:0/1296043977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e40c6f50 0x7fb8e4156320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 >> 172.21.15.133:0/1296043977 conn(0x7fb8e4062000 msgr2=0x7fb8e4159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:31.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 shutdown_connections 2024-02-21T23:53:31.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.186+0000 7fb8e942d640 1 -- 172.21.15.133:0/1296043977 wait complete. 2024-02-21T23:53:31.192 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":873,"stamp":"2024-02-21T23:53:31.042599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":41122540,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":1110,"num_read_kb":1160,"num_write":15803,"num_write_kb":371997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16682,"ondisk_log_size":16682,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":426808,"kb_used_data":56632,"kb_used_omap":12,"kb_used_meta":370099,"kb_avail":754547912,"statfs":{"total":773094113280,"available":772657061888,"internally_reserved":0,"allocated":57991168,"data_stored":83162120,"data_compressed":7608710,"data_compressed_allocated":11812864,"data_compressed_original":38363136,"omap_allocated":12750,"internal_metadata":378981938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14811,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":15,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002580"},"pg_stats":[{"pgid":"2.1f","version":"26'604","reported_seq":643,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:17.860114+0000","last_change":"2024-02-21T23:25:16.134390+0000","last_active":"2024-02-21T23:53:17.860114+0000","last_peered":"2024-02-21T23:53:17.860114+0000","last_clean":"2024-02-21T23:53:17.860114+0000","last_became_active":"2024-02-21T23:25:16.134112+0000","last_became_peered":"2024-02-21T23:25:16.134112+0000","last_unstale":"2024-02-21T23:53:17.860114+0000","last_undegraded":"2024-02-21T23:53:17.860114+0000","last_fullsized":"2024-02-21T23:53:17.860114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'30","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.462957+0000","last_change":"2024-02-21T23:25:17.135738+0000","last_active":"2024-02-21T23:53:24.462957+0000","last_peered":"2024-02-21T23:53:24.462957+0000","last_clean":"2024-02-21T23:53:24.462957+0000","last_became_active":"2024-02-21T23:25:17.135586+0000","last_became_peered":"2024-02-21T23:25:17.135586+0000","last_unstale":"2024-02-21T23:53:24.462957+0000","last_undegraded":"2024-02-21T23:53:24.462957+0000","last_fullsized":"2024-02-21T23:53:24.462957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'161","reported_seq":190,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.384215+0000","last_change":"2024-02-21T23:25:22.248962+0000","last_active":"2024-02-21T23:52:56.384215+0000","last_peered":"2024-02-21T23:52:56.384215+0000","last_clean":"2024-02-21T23:52:56.384215+0000","last_became_active":"2024-02-21T23:25:22.248610+0000","last_became_peered":"2024-02-21T23:25:22.248610+0000","last_unstale":"2024-02-21T23:52:56.384215+0000","last_undegraded":"2024-02-21T23:52:56.384215+0000","last_fullsized":"2024-02-21T23:52:56.384215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:45.939036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":10997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'750","reported_seq":790,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:05.342762+0000","last_change":"2024-02-21T23:25:16.123635+0000","last_active":"2024-02-21T23:53:05.342762+0000","last_peered":"2024-02-21T23:53:05.342762+0000","last_clean":"2024-02-21T23:53:05.342762+0000","last_became_active":"2024-02-21T23:25:16.123374+0000","last_became_peered":"2024-02-21T23:25:16.123374+0000","last_unstale":"2024-02-21T23:53:05.342762+0000","last_undegraded":"2024-02-21T23:53:05.342762+0000","last_fullsized":"2024-02-21T23:53:05.342762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":4534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604770+0000","last_change":"2024-02-21T23:25:17.192858+0000","last_active":"2024-02-21T23:53:24.604770+0000","last_peered":"2024-02-21T23:53:24.604770+0000","last_clean":"2024-02-21T23:53:24.604770+0000","last_became_active":"2024-02-21T23:25:17.192626+0000","last_became_peered":"2024-02-21T23:25:17.192626+0000","last_unstale":"2024-02-21T23:53:24.604770+0000","last_undegraded":"2024-02-21T23:53:24.604770+0000","last_fullsized":"2024-02-21T23:53:24.604770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'132","reported_seq":168,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.423235+0000","last_change":"2024-02-21T23:25:57.583837+0000","last_active":"2024-02-21T23:52:56.423235+0000","last_peered":"2024-02-21T23:52:56.423235+0000","last_clean":"2024-02-21T23:52:56.423235+0000","last_became_active":"2024-02-21T23:25:57.583416+0000","last_became_peered":"2024-02-21T23:25:57.583416+0000","last_unstale":"2024-02-21T23:52:56.423235+0000","last_undegraded":"2024-02-21T23:52:56.423235+0000","last_fullsized":"2024-02-21T23:52:56.423235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:25:24.780185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":6267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"26'1343","reported_seq":1380,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.892783+0000","last_change":"2024-02-21T23:25:16.134805+0000","last_active":"2024-02-21T23:53:15.892783+0000","last_peered":"2024-02-21T23:53:15.892783+0000","last_clean":"2024-02-21T23:53:15.892783+0000","last_became_active":"2024-02-21T23:25:16.134533+0000","last_became_peered":"2024-02-21T23:25:16.134533+0000","last_unstale":"2024-02-21T23:53:15.892783+0000","last_undegraded":"2024-02-21T23:53:15.892783+0000","last_fullsized":"2024-02-21T23:53:15.892783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1334,"num_write_kb":8974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'32","reported_seq":67,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494748+0000","last_change":"2024-02-21T23:25:17.182131+0000","last_active":"2024-02-21T23:53:24.494748+0000","last_peered":"2024-02-21T23:53:24.494748+0000","last_clean":"2024-02-21T23:53:24.494748+0000","last_became_active":"2024-02-21T23:25:17.181886+0000","last_became_peered":"2024-02-21T23:25:17.181886+0000","last_unstale":"2024-02-21T23:53:24.494748+0000","last_undegraded":"2024-02-21T23:53:24.494748+0000","last_fullsized":"2024-02-21T23:53:24.494748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'141","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.423026+0000","last_change":"2024-02-21T23:25:21.683506+0000","last_active":"2024-02-21T23:52:56.423026+0000","last_peered":"2024-02-21T23:52:56.423026+0000","last_clean":"2024-02-21T23:52:56.423026+0000","last_became_active":"2024-02-21T23:25:21.683115+0000","last_became_peered":"2024-02-21T23:25:21.683115+0000","last_unstale":"2024-02-21T23:52:56.423026+0000","last_undegraded":"2024-02-21T23:52:56.423026+0000","last_fullsized":"2024-02-21T23:52:56.423026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:13.865145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":12261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"26'6","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:08.503461+0000","last_change":"2024-02-21T23:25:16.130851+0000","last_active":"2024-02-21T23:53:08.503461+0000","last_peered":"2024-02-21T23:53:08.503461+0000","last_clean":"2024-02-21T23:53:08.503461+0000","last_became_active":"2024-02-21T23:25:16.130604+0000","last_became_peered":"2024-02-21T23:25:16.130604+0000","last_unstale":"2024-02-21T23:53:08.503461+0000","last_undegraded":"2024-02-21T23:53:08.503461+0000","last_fullsized":"2024-02-21T23:53:08.503461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'47","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.495013+0000","last_change":"2024-02-21T23:25:17.192642+0000","last_active":"2024-02-21T23:53:24.495013+0000","last_peered":"2024-02-21T23:53:24.495013+0000","last_clean":"2024-02-21T23:53:24.495013+0000","last_became_active":"2024-02-21T23:25:17.192394+0000","last_became_peered":"2024-02-21T23:25:17.192394+0000","last_unstale":"2024-02-21T23:53:24.495013+0000","last_undegraded":"2024-02-21T23:53:24.495013+0000","last_fullsized":"2024-02-21T23:53:24.495013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'145","reported_seq":174,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:53.583749+0000","last_change":"2024-02-21T23:25:21.679491+0000","last_active":"2024-02-21T23:52:53.583749+0000","last_peered":"2024-02-21T23:52:53.583749+0000","last_clean":"2024-02-21T23:52:53.583749+0000","last_became_active":"2024-02-21T23:25:21.679117+0000","last_became_peered":"2024-02-21T23:25:21.679117+0000","last_unstale":"2024-02-21T23:52:53.583749+0000","last_undegraded":"2024-02-21T23:52:53.583749+0000","last_fullsized":"2024-02-21T23:52:53.583749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:52.680971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":9573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626233+0000","last_change":"2024-02-21T23:25:17.161438+0000","last_active":"2024-02-21T23:53:24.626233+0000","last_peered":"2024-02-21T23:53:24.626233+0000","last_clean":"2024-02-21T23:53:24.626233+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:24.626233+0000","last_undegraded":"2024-02-21T23:53:24.626233+0000","last_fullsized":"2024-02-21T23:53:24.626233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'648","reported_seq":686,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:07.478231+0000","last_change":"2024-02-21T23:25:16.136913+0000","last_active":"2024-02-21T23:53:07.478231+0000","last_peered":"2024-02-21T23:53:07.478231+0000","last_clean":"2024-02-21T23:53:07.478231+0000","last_became_active":"2024-02-21T23:25:16.136628+0000","last_became_peered":"2024-02-21T23:25:16.136628+0000","last_unstale":"2024-02-21T23:53:07.478231+0000","last_undegraded":"2024-02-21T23:53:07.478231+0000","last_fullsized":"2024-02-21T23:53:07.478231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":644,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'134","reported_seq":171,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.695314+0000","last_change":"2024-02-21T23:25:57.526324+0000","last_active":"2024-02-21T23:52:56.695314+0000","last_peered":"2024-02-21T23:52:56.695314+0000","last_clean":"2024-02-21T23:52:56.695314+0000","last_became_active":"2024-02-21T23:25:57.525854+0000","last_became_peered":"2024-02-21T23:25:57.525854+0000","last_unstale":"2024-02-21T23:52:56.695314+0000","last_undegraded":"2024-02-21T23:52:56.695314+0000","last_fullsized":"2024-02-21T23:52:56.695314+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:12:32.356243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.465980+0000","last_change":"2024-02-21T23:25:17.137829+0000","last_active":"2024-02-21T23:53:24.465980+0000","last_peered":"2024-02-21T23:53:24.465980+0000","last_clean":"2024-02-21T23:53:24.465980+0000","last_became_active":"2024-02-21T23:25:17.137557+0000","last_became_peered":"2024-02-21T23:25:17.137557+0000","last_unstale":"2024-02-21T23:53:24.465980+0000","last_undegraded":"2024-02-21T23:53:24.465980+0000","last_fullsized":"2024-02-21T23:53:24.465980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:59.099888+0000","last_change":"2024-02-21T23:25:16.146181+0000","last_active":"2024-02-21T23:52:59.099888+0000","last_peered":"2024-02-21T23:52:59.099888+0000","last_clean":"2024-02-21T23:52:59.099888+0000","last_became_active":"2024-02-21T23:25:16.145932+0000","last_became_peered":"2024-02-21T23:25:16.145932+0000","last_unstale":"2024-02-21T23:52:59.099888+0000","last_undegraded":"2024-02-21T23:52:59.099888+0000","last_fullsized":"2024-02-21T23:52:59.099888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'165","reported_seq":195,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.095405+0000","last_change":"2024-02-21T23:25:21.686044+0000","last_active":"2024-02-21T23:52:55.095405+0000","last_peered":"2024-02-21T23:52:55.095405+0000","last_clean":"2024-02-21T23:52:55.095405+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:52:55.095405+0000","last_undegraded":"2024-02-21T23:52:55.095405+0000","last_fullsized":"2024-02-21T23:52:55.095405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:39.740454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'37","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471561+0000","last_change":"2024-02-21T23:25:17.190236+0000","last_active":"2024-02-21T23:53:24.471561+0000","last_peered":"2024-02-21T23:53:24.471561+0000","last_clean":"2024-02-21T23:53:24.471561+0000","last_became_active":"2024-02-21T23:25:17.189957+0000","last_became_peered":"2024-02-21T23:25:17.189957+0000","last_unstale":"2024-02-21T23:53:24.471561+0000","last_undegraded":"2024-02-21T23:53:24.471561+0000","last_fullsized":"2024-02-21T23:53:24.471561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:03.326866+0000","last_change":"2024-02-21T23:25:16.148108+0000","last_active":"2024-02-21T23:53:03.326866+0000","last_peered":"2024-02-21T23:53:03.326866+0000","last_clean":"2024-02-21T23:53:03.326866+0000","last_became_active":"2024-02-21T23:25:16.147930+0000","last_became_peered":"2024-02-21T23:25:16.147930+0000","last_unstale":"2024-02-21T23:53:03.326866+0000","last_undegraded":"2024-02-21T23:53:03.326866+0000","last_fullsized":"2024-02-21T23:53:03.326866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'132","reported_seq":162,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.410926+0000","last_change":"2024-02-21T23:25:22.248150+0000","last_active":"2024-02-21T23:52:56.410926+0000","last_peered":"2024-02-21T23:52:56.410926+0000","last_clean":"2024-02-21T23:52:56.410926+0000","last_became_active":"2024-02-21T23:25:22.247769+0000","last_became_peered":"2024-02-21T23:25:22.247769+0000","last_unstale":"2024-02-21T23:52:56.410926+0000","last_undegraded":"2024-02-21T23:52:56.410926+0000","last_fullsized":"2024-02-21T23:52:56.410926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"26'34","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.477731+0000","last_change":"2024-02-21T23:25:17.189758+0000","last_active":"2024-02-21T23:53:24.477731+0000","last_peered":"2024-02-21T23:53:24.477731+0000","last_clean":"2024-02-21T23:53:24.477731+0000","last_became_active":"2024-02-21T23:25:17.189492+0000","last_became_peered":"2024-02-21T23:25:17.189492+0000","last_unstale":"2024-02-21T23:53:24.477731+0000","last_undegraded":"2024-02-21T23:53:24.477731+0000","last_fullsized":"2024-02-21T23:53:24.477731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:42.526598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'7","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:17.858179+0000","last_change":"2024-02-21T23:25:16.131611+0000","last_active":"2024-02-21T23:53:17.858179+0000","last_peered":"2024-02-21T23:53:17.858179+0000","last_clean":"2024-02-21T23:53:17.858179+0000","last_became_active":"2024-02-21T23:25:16.131158+0000","last_became_peered":"2024-02-21T23:25:16.131158+0000","last_unstale":"2024-02-21T23:53:17.858179+0000","last_undegraded":"2024-02-21T23:53:17.858179+0000","last_fullsized":"2024-02-21T23:53:17.858179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'116","reported_seq":146,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.014642+0000","last_change":"2024-02-21T23:25:21.683821+0000","last_active":"2024-02-21T23:52:52.014642+0000","last_peered":"2024-02-21T23:52:52.014642+0000","last_clean":"2024-02-21T23:52:52.014642+0000","last_became_active":"2024-02-21T23:25:21.683466+0000","last_became_peered":"2024-02-21T23:25:21.683466+0000","last_unstale":"2024-02-21T23:52:52.014642+0000","last_undegraded":"2024-02-21T23:52:52.014642+0000","last_fullsized":"2024-02-21T23:52:52.014642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":7338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"26'38","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.602687+0000","last_change":"2024-02-21T23:25:17.191394+0000","last_active":"2024-02-21T23:53:24.602687+0000","last_peered":"2024-02-21T23:53:24.602687+0000","last_clean":"2024-02-21T23:53:24.602687+0000","last_became_active":"2024-02-21T23:25:17.191106+0000","last_became_peered":"2024-02-21T23:25:17.191106+0000","last_unstale":"2024-02-21T23:53:24.602687+0000","last_undegraded":"2024-02-21T23:53:24.602687+0000","last_fullsized":"2024-02-21T23:53:24.602687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'9","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.843551+0000","last_change":"2024-02-21T23:25:16.134862+0000","last_active":"2024-02-21T23:53:18.843551+0000","last_peered":"2024-02-21T23:53:18.843551+0000","last_clean":"2024-02-21T23:53:18.843551+0000","last_became_active":"2024-02-21T23:25:16.134602+0000","last_became_peered":"2024-02-21T23:25:16.134602+0000","last_unstale":"2024-02-21T23:53:18.843551+0000","last_undegraded":"2024-02-21T23:53:18.843551+0000","last_fullsized":"2024-02-21T23:53:18.843551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'143","reported_seq":174,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.422758+0000","last_change":"2024-02-21T23:25:22.249606+0000","last_active":"2024-02-21T23:52:56.422758+0000","last_peered":"2024-02-21T23:52:56.422758+0000","last_clean":"2024-02-21T23:52:56.422758+0000","last_became_active":"2024-02-21T23:25:22.249214+0000","last_became_peered":"2024-02-21T23:25:22.249214+0000","last_unstale":"2024-02-21T23:52:56.422758+0000","last_undegraded":"2024-02-21T23:52:56.422758+0000","last_fullsized":"2024-02-21T23:52:56.422758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:36.929076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":6395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'33","reported_seq":70,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471655+0000","last_change":"2024-02-21T23:25:17.183290+0000","last_active":"2024-02-21T23:53:24.471655+0000","last_peered":"2024-02-21T23:53:24.471655+0000","last_clean":"2024-02-21T23:53:24.471655+0000","last_became_active":"2024-02-21T23:25:17.183047+0000","last_became_peered":"2024-02-21T23:25:17.183047+0000","last_unstale":"2024-02-21T23:53:24.471655+0000","last_undegraded":"2024-02-21T23:53:24.471655+0000","last_fullsized":"2024-02-21T23:53:24.471655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'1368","reported_seq":1407,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.000230+0000","last_change":"2024-02-21T23:25:16.146202+0000","last_active":"2024-02-21T23:53:23.000230+0000","last_peered":"2024-02-21T23:53:23.000230+0000","last_clean":"2024-02-21T23:53:23.000230+0000","last_became_active":"2024-02-21T23:25:16.145911+0000","last_became_peered":"2024-02-21T23:25:16.145911+0000","last_unstale":"2024-02-21T23:53:23.000230+0000","last_undegraded":"2024-02-21T23:53:23.000230+0000","last_fullsized":"2024-02-21T23:53:23.000230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3511,"num_write_kb":1242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'135","reported_seq":167,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700303+0000","last_change":"2024-02-21T23:25:22.243297+0000","last_active":"2024-02-21T23:52:56.700303+0000","last_peered":"2024-02-21T23:52:56.700303+0000","last_clean":"2024-02-21T23:52:56.700303+0000","last_became_active":"2024-02-21T23:25:22.242922+0000","last_became_peered":"2024-02-21T23:25:22.242922+0000","last_unstale":"2024-02-21T23:52:56.700303+0000","last_undegraded":"2024-02-21T23:52:56.700303+0000","last_fullsized":"2024-02-21T23:52:56.700303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:52:51.773574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":7447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"26'37","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.456470+0000","last_change":"2024-02-21T23:25:17.190577+0000","last_active":"2024-02-21T23:53:24.456470+0000","last_peered":"2024-02-21T23:53:24.456470+0000","last_clean":"2024-02-21T23:53:24.456470+0000","last_became_active":"2024-02-21T23:25:17.190332+0000","last_became_peered":"2024-02-21T23:25:17.190332+0000","last_unstale":"2024-02-21T23:53:24.456470+0000","last_undegraded":"2024-02-21T23:53:24.456470+0000","last_fullsized":"2024-02-21T23:53:24.456470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'3","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.853192+0000","last_change":"2024-02-21T23:25:16.134076+0000","last_active":"2024-02-21T23:53:15.853192+0000","last_peered":"2024-02-21T23:53:15.853192+0000","last_clean":"2024-02-21T23:53:15.853192+0000","last_became_active":"2024-02-21T23:25:16.133816+0000","last_became_peered":"2024-02-21T23:25:16.133816+0000","last_unstale":"2024-02-21T23:53:15.853192+0000","last_undegraded":"2024-02-21T23:53:15.853192+0000","last_fullsized":"2024-02-21T23:53:15.853192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'127","reported_seq":158,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.466825+0000","last_change":"2024-02-21T23:25:22.243360+0000","last_active":"2024-02-21T23:52:48.466825+0000","last_peered":"2024-02-21T23:52:48.466825+0000","last_clean":"2024-02-21T23:52:48.466825+0000","last_became_active":"2024-02-21T23:25:22.242958+0000","last_became_peered":"2024-02-21T23:25:22.242958+0000","last_unstale":"2024-02-21T23:52:48.466825+0000","last_undegraded":"2024-02-21T23:52:48.466825+0000","last_fullsized":"2024-02-21T23:52:48.466825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":7411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494810+0000","last_change":"2024-02-21T23:25:17.192402+0000","last_active":"2024-02-21T23:53:24.494810+0000","last_peered":"2024-02-21T23:53:24.494810+0000","last_clean":"2024-02-21T23:53:24.494810+0000","last_became_active":"2024-02-21T23:25:17.192138+0000","last_became_peered":"2024-02-21T23:25:17.192138+0000","last_unstale":"2024-02-21T23:53:24.494810+0000","last_undegraded":"2024-02-21T23:53:24.494810+0000","last_fullsized":"2024-02-21T23:53:24.494810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:03.326742+0000","last_change":"2024-02-21T23:25:16.145521+0000","last_active":"2024-02-21T23:53:03.326742+0000","last_peered":"2024-02-21T23:53:03.326742+0000","last_clean":"2024-02-21T23:53:03.326742+0000","last_became_active":"2024-02-21T23:25:16.145255+0000","last_became_peered":"2024-02-21T23:25:16.145255+0000","last_unstale":"2024-02-21T23:53:03.326742+0000","last_undegraded":"2024-02-21T23:53:03.326742+0000","last_fullsized":"2024-02-21T23:53:03.326742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'208","reported_seq":239,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.413727+0000","last_change":"2024-02-21T23:25:21.683849+0000","last_active":"2024-02-21T23:52:56.413727+0000","last_peered":"2024-02-21T23:52:56.413727+0000","last_clean":"2024-02-21T23:52:56.413727+0000","last_became_active":"2024-02-21T23:25:21.683379+0000","last_became_peered":"2024-02-21T23:25:21.683379+0000","last_unstale":"2024-02-21T23:52:56.413727+0000","last_undegraded":"2024-02-21T23:52:56.413727+0000","last_fullsized":"2024-02-21T23:52:56.413727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":191,"num_write_kb":16487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"26'31","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.482603+0000","last_change":"2024-02-21T23:25:17.166232+0000","last_active":"2024-02-21T23:53:24.482603+0000","last_peered":"2024-02-21T23:53:24.482603+0000","last_clean":"2024-02-21T23:53:24.482603+0000","last_became_active":"2024-02-21T23:25:17.165999+0000","last_became_peered":"2024-02-21T23:25:17.165999+0000","last_unstale":"2024-02-21T23:53:24.482603+0000","last_undegraded":"2024-02-21T23:53:24.482603+0000","last_fullsized":"2024-02-21T23:53:24.482603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:45.125986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:23.691688+0000","last_change":"2024-02-21T23:25:16.147082+0000","last_active":"2024-02-21T23:52:23.691688+0000","last_peered":"2024-02-21T23:52:23.691688+0000","last_clean":"2024-02-21T23:52:23.691688+0000","last_became_active":"2024-02-21T23:25:16.146797+0000","last_became_peered":"2024-02-21T23:25:16.146797+0000","last_unstale":"2024-02-21T23:52:23.691688+0000","last_undegraded":"2024-02-21T23:52:23.691688+0000","last_fullsized":"2024-02-21T23:52:23.691688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'125","reported_seq":150,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.407469+0000","last_change":"2024-02-21T23:25:57.583768+0000","last_active":"2024-02-21T23:52:56.407469+0000","last_peered":"2024-02-21T23:52:56.407469+0000","last_clean":"2024-02-21T23:52:56.407469+0000","last_became_active":"2024-02-21T23:25:57.583297+0000","last_became_peered":"2024-02-21T23:25:57.583297+0000","last_unstale":"2024-02-21T23:52:56.407469+0000","last_undegraded":"2024-02-21T23:52:56.407469+0000","last_fullsized":"2024-02-21T23:52:56.407469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:19:14.798657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":6113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,6],"acting":[0,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"26'33","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.454823+0000","last_change":"2024-02-21T23:25:17.190425+0000","last_active":"2024-02-21T23:53:24.454823+0000","last_peered":"2024-02-21T23:53:24.454823+0000","last_clean":"2024-02-21T23:53:24.454823+0000","last_became_active":"2024-02-21T23:25:17.190172+0000","last_became_peered":"2024-02-21T23:25:17.190172+0000","last_unstale":"2024-02-21T23:53:24.454823+0000","last_undegraded":"2024-02-21T23:53:24.454823+0000","last_fullsized":"2024-02-21T23:53:24.454823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'999","reported_seq":2032,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.855182+0000","last_change":"2024-02-21T23:25:16.108263+0000","last_active":"2024-02-21T23:53:23.855182+0000","last_peered":"2024-02-21T23:53:23.855182+0000","last_clean":"2024-02-21T23:53:23.855182+0000","last_became_active":"2024-02-21T23:25:16.107822+0000","last_became_peered":"2024-02-21T23:25:16.107822+0000","last_unstale":"2024-02-21T23:53:23.855182+0000","last_undegraded":"2024-02-21T23:53:23.855182+0000","last_fullsized":"2024-02-21T23:53:23.855182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":995,"num_write_kb":1061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'134","reported_seq":164,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.702471+0000","last_change":"2024-02-21T23:25:21.686034+0000","last_active":"2024-02-21T23:52:56.702471+0000","last_peered":"2024-02-21T23:52:56.702471+0000","last_clean":"2024-02-21T23:52:56.702471+0000","last_became_active":"2024-02-21T23:25:21.685546+0000","last_became_peered":"2024-02-21T23:25:21.685546+0000","last_unstale":"2024-02-21T23:52:56.702471+0000","last_undegraded":"2024-02-21T23:52:56.702471+0000","last_fullsized":"2024-02-21T23:52:56.702471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":10678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607087+0000","last_change":"2024-02-21T23:25:17.161738+0000","last_active":"2024-02-21T23:53:24.607087+0000","last_peered":"2024-02-21T23:53:24.607087+0000","last_clean":"2024-02-21T23:53:24.607087+0000","last_became_active":"2024-02-21T23:25:17.161459+0000","last_became_peered":"2024-02-21T23:25:17.161459+0000","last_unstale":"2024-02-21T23:53:24.607087+0000","last_undegraded":"2024-02-21T23:53:24.607087+0000","last_fullsized":"2024-02-21T23:53:24.607087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:31:58.434561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'541","reported_seq":581,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.846835+0000","last_change":"2024-02-21T23:25:16.147162+0000","last_active":"2024-02-21T23:53:18.846835+0000","last_peered":"2024-02-21T23:53:18.846835+0000","last_clean":"2024-02-21T23:53:18.846835+0000","last_became_active":"2024-02-21T23:25:16.146850+0000","last_became_peered":"2024-02-21T23:25:16.146850+0000","last_unstale":"2024-02-21T23:53:18.846835+0000","last_undegraded":"2024-02-21T23:53:18.846835+0000","last_fullsized":"2024-02-21T23:53:18.846835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'140","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.987450+0000","last_change":"2024-02-21T23:25:22.248109+0000","last_active":"2024-02-21T23:52:52.987450+0000","last_peered":"2024-02-21T23:52:52.987450+0000","last_clean":"2024-02-21T23:52:52.987450+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:52:52.987450+0000","last_undegraded":"2024-02-21T23:52:52.987450+0000","last_fullsized":"2024-02-21T23:52:52.987450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"26'35","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.466335+0000","last_change":"2024-02-21T23:25:57.600968+0000","last_active":"2024-02-21T23:53:24.466335+0000","last_peered":"2024-02-21T23:53:24.466335+0000","last_clean":"2024-02-21T23:53:24.466335+0000","last_became_active":"2024-02-21T23:25:57.600644+0000","last_became_peered":"2024-02-21T23:25:57.600644+0000","last_unstale":"2024-02-21T23:53:24.466335+0000","last_undegraded":"2024-02-21T23:53:24.466335+0000","last_fullsized":"2024-02-21T23:53:24.466335+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:13:09.640854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'5","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:10.587650+0000","last_change":"2024-02-21T23:25:16.127688+0000","last_active":"2024-02-21T23:53:10.587650+0000","last_peered":"2024-02-21T23:53:10.587650+0000","last_clean":"2024-02-21T23:53:10.587650+0000","last_became_active":"2024-02-21T23:25:16.127388+0000","last_became_peered":"2024-02-21T23:25:16.127388+0000","last_unstale":"2024-02-21T23:53:10.587650+0000","last_undegraded":"2024-02-21T23:53:10.587650+0000","last_fullsized":"2024-02-21T23:53:10.587650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:51:13.500334+0000","last_change":"2024-02-21T23:25:05.612817+0000","last_active":"2024-02-21T23:51:13.500334+0000","last_peered":"2024-02-21T23:51:13.500334+0000","last_clean":"2024-02-21T23:51:13.500334+0000","last_became_active":"2024-02-21T23:25:05.612483+0000","last_became_peered":"2024-02-21T23:25:05.612483+0000","last_unstale":"2024-02-21T23:51:13.500334+0000","last_undegraded":"2024-02-21T23:51:13.500334+0000","last_fullsized":"2024-02-21T23:51:13.500334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_clean_scrub_stamp":"2024-02-21T23:25:03.736732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:23:33.073040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'168","reported_seq":199,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.098108+0000","last_change":"2024-02-21T23:25:22.248174+0000","last_active":"2024-02-21T23:52:55.098108+0000","last_peered":"2024-02-21T23:52:55.098108+0000","last_clean":"2024-02-21T23:52:55.098108+0000","last_became_active":"2024-02-21T23:25:22.247834+0000","last_became_peered":"2024-02-21T23:25:22.247834+0000","last_unstale":"2024-02-21T23:52:55.098108+0000","last_undegraded":"2024-02-21T23:52:55.098108+0000","last_fullsized":"2024-02-21T23:52:55.098108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:13.953358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":10993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604646+0000","last_change":"2024-02-21T23:25:17.192575+0000","last_active":"2024-02-21T23:53:24.604646+0000","last_peered":"2024-02-21T23:53:24.604646+0000","last_clean":"2024-02-21T23:53:24.604646+0000","last_became_active":"2024-02-21T23:25:17.192293+0000","last_became_peered":"2024-02-21T23:25:17.192293+0000","last_unstale":"2024-02-21T23:53:24.604646+0000","last_undegraded":"2024-02-21T23:53:24.604646+0000","last_fullsized":"2024-02-21T23:53:24.604646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:57.046912+0000","last_change":"2024-02-21T23:25:16.128369+0000","last_active":"2024-02-21T23:52:57.046912+0000","last_peered":"2024-02-21T23:52:57.046912+0000","last_clean":"2024-02-21T23:52:57.046912+0000","last_became_active":"2024-02-21T23:25:16.128067+0000","last_became_peered":"2024-02-21T23:25:16.128067+0000","last_unstale":"2024-02-21T23:52:57.046912+0000","last_undegraded":"2024-02-21T23:52:57.046912+0000","last_fullsized":"2024-02-21T23:52:57.046912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'138","reported_seq":168,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.670157+0000","last_change":"2024-02-21T23:25:22.243282+0000","last_active":"2024-02-21T23:52:52.670157+0000","last_peered":"2024-02-21T23:52:52.670157+0000","last_clean":"2024-02-21T23:52:52.670157+0000","last_became_active":"2024-02-21T23:25:22.242916+0000","last_became_peered":"2024-02-21T23:25:22.242916+0000","last_unstale":"2024-02-21T23:52:52.670157+0000","last_undegraded":"2024-02-21T23:52:52.670157+0000","last_fullsized":"2024-02-21T23:52:52.670157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"26'40","reported_seq":76,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493486+0000","last_change":"2024-02-21T23:25:17.185418+0000","last_active":"2024-02-21T23:53:24.493486+0000","last_peered":"2024-02-21T23:53:24.493486+0000","last_clean":"2024-02-21T23:53:24.493486+0000","last_became_active":"2024-02-21T23:25:17.185119+0000","last_became_peered":"2024-02-21T23:25:17.185119+0000","last_unstale":"2024-02-21T23:53:24.493486+0000","last_undegraded":"2024-02-21T23:53:24.493486+0000","last_fullsized":"2024-02-21T23:53:24.493486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:18.471011+0000","last_change":"2024-02-21T23:25:16.151088+0000","last_active":"2024-02-21T23:52:18.471011+0000","last_peered":"2024-02-21T23:52:18.471011+0000","last_clean":"2024-02-21T23:52:18.471011+0000","last_became_active":"2024-02-21T23:25:16.150819+0000","last_became_peered":"2024-02-21T23:25:16.150819+0000","last_unstale":"2024-02-21T23:52:18.471011+0000","last_undegraded":"2024-02-21T23:52:18.471011+0000","last_fullsized":"2024-02-21T23:52:18.471011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'189","reported_seq":219,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.685417+0000","last_change":"2024-02-21T23:25:21.684592+0000","last_active":"2024-02-21T23:52:56.685417+0000","last_peered":"2024-02-21T23:52:56.685417+0000","last_clean":"2024-02-21T23:52:56.685417+0000","last_became_active":"2024-02-21T23:25:21.684192+0000","last_became_peered":"2024-02-21T23:25:21.684192+0000","last_unstale":"2024-02-21T23:52:56.685417+0000","last_undegraded":"2024-02-21T23:52:56.685417+0000","last_fullsized":"2024-02-21T23:52:56.685417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":12683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.606181+0000","last_change":"2024-02-21T23:25:17.184268+0000","last_active":"2024-02-21T23:53:24.606181+0000","last_peered":"2024-02-21T23:53:24.606181+0000","last_clean":"2024-02-21T23:53:24.606181+0000","last_became_active":"2024-02-21T23:25:17.184027+0000","last_became_peered":"2024-02-21T23:25:17.184027+0000","last_unstale":"2024-02-21T23:53:24.606181+0000","last_undegraded":"2024-02-21T23:53:24.606181+0000","last_fullsized":"2024-02-21T23:53:24.606181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'4","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:09.554814+0000","last_change":"2024-02-21T23:25:16.108099+0000","last_active":"2024-02-21T23:53:09.554814+0000","last_peered":"2024-02-21T23:53:09.554814+0000","last_clean":"2024-02-21T23:53:09.554814+0000","last_became_active":"2024-02-21T23:25:16.107815+0000","last_became_peered":"2024-02-21T23:25:16.107815+0000","last_unstale":"2024-02-21T23:53:09.554814+0000","last_undegraded":"2024-02-21T23:53:09.554814+0000","last_fullsized":"2024-02-21T23:53:09.554814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'131","reported_seq":161,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.464577+0000","last_change":"2024-02-21T23:25:22.248926+0000","last_active":"2024-02-21T23:52:48.464577+0000","last_peered":"2024-02-21T23:52:48.464577+0000","last_clean":"2024-02-21T23:52:48.464577+0000","last_became_active":"2024-02-21T23:25:22.248558+0000","last_became_peered":"2024-02-21T23:25:22.248558+0000","last_unstale":"2024-02-21T23:52:48.464577+0000","last_undegraded":"2024-02-21T23:52:48.464577+0000","last_fullsized":"2024-02-21T23:52:48.464577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"26'23","reported_seq":59,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605888+0000","last_change":"2024-02-21T23:25:17.183254+0000","last_active":"2024-02-21T23:53:24.605888+0000","last_peered":"2024-02-21T23:53:24.605888+0000","last_clean":"2024-02-21T23:53:24.605888+0000","last_became_active":"2024-02-21T23:25:17.183020+0000","last_became_peered":"2024-02-21T23:25:17.183020+0000","last_unstale":"2024-02-21T23:53:24.605888+0000","last_undegraded":"2024-02-21T23:53:24.605888+0000","last_fullsized":"2024-02-21T23:53:24.605888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:23.581289+0000","last_change":"2024-02-21T23:25:16.129131+0000","last_active":"2024-02-21T23:52:23.581289+0000","last_peered":"2024-02-21T23:52:23.581289+0000","last_clean":"2024-02-21T23:52:23.581289+0000","last_became_active":"2024-02-21T23:25:16.128853+0000","last_became_peered":"2024-02-21T23:25:16.128853+0000","last_unstale":"2024-02-21T23:52:23.581289+0000","last_undegraded":"2024-02-21T23:52:23.581289+0000","last_fullsized":"2024-02-21T23:52:23.581289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:29.302181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'163","reported_seq":193,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.422752+0000","last_change":"2024-02-21T23:25:21.684342+0000","last_active":"2024-02-21T23:52:56.422752+0000","last_peered":"2024-02-21T23:52:56.422752+0000","last_clean":"2024-02-21T23:52:56.422752+0000","last_became_active":"2024-02-21T23:25:21.683991+0000","last_became_peered":"2024-02-21T23:25:21.683991+0000","last_unstale":"2024-02-21T23:52:56.422752+0000","last_undegraded":"2024-02-21T23:52:56.422752+0000","last_fullsized":"2024-02-21T23:52:56.422752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:21:00.312314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":12715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"26'37","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.606217+0000","last_change":"2024-02-21T23:25:17.183442+0000","last_active":"2024-02-21T23:53:24.606217+0000","last_peered":"2024-02-21T23:53:24.606217+0000","last_clean":"2024-02-21T23:53:24.606217+0000","last_became_active":"2024-02-21T23:25:17.183057+0000","last_became_peered":"2024-02-21T23:25:17.183057+0000","last_unstale":"2024-02-21T23:53:24.606217+0000","last_undegraded":"2024-02-21T23:53:24.606217+0000","last_fullsized":"2024-02-21T23:53:24.606217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'13","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.623678+0000","last_change":"2024-02-21T23:25:16.145801+0000","last_active":"2024-02-21T23:53:18.623678+0000","last_peered":"2024-02-21T23:53:18.623678+0000","last_clean":"2024-02-21T23:53:18.623678+0000","last_became_active":"2024-02-21T23:25:16.145523+0000","last_became_peered":"2024-02-21T23:25:16.145523+0000","last_unstale":"2024-02-21T23:53:18.623678+0000","last_undegraded":"2024-02-21T23:53:18.623678+0000","last_fullsized":"2024-02-21T23:53:18.623678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'157","reported_seq":186,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.093033+0000","last_change":"2024-02-21T23:25:21.679191+0000","last_active":"2024-02-21T23:52:55.093033+0000","last_peered":"2024-02-21T23:52:55.093033+0000","last_clean":"2024-02-21T23:52:55.093033+0000","last_became_active":"2024-02-21T23:25:21.678715+0000","last_became_peered":"2024-02-21T23:25:21.678715+0000","last_unstale":"2024-02-21T23:52:55.093033+0000","last_undegraded":"2024-02-21T23:52:55.093033+0000","last_fullsized":"2024-02-21T23:52:55.093033+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":11459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"26'29","reported_seq":65,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.625866+0000","last_change":"2024-02-21T23:25:17.138401+0000","last_active":"2024-02-21T23:53:24.625866+0000","last_peered":"2024-02-21T23:53:24.625866+0000","last_clean":"2024-02-21T23:53:24.625866+0000","last_became_active":"2024-02-21T23:25:17.138136+0000","last_became_peered":"2024-02-21T23:25:17.138136+0000","last_unstale":"2024-02-21T23:53:24.625866+0000","last_undegraded":"2024-02-21T23:53:24.625866+0000","last_fullsized":"2024-02-21T23:53:24.625866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'8","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.890247+0000","last_change":"2024-02-21T23:25:16.130209+0000","last_active":"2024-02-21T23:53:15.890247+0000","last_peered":"2024-02-21T23:53:15.890247+0000","last_clean":"2024-02-21T23:53:15.890247+0000","last_became_active":"2024-02-21T23:25:16.129974+0000","last_became_peered":"2024-02-21T23:25:16.129974+0000","last_unstale":"2024-02-21T23:53:15.890247+0000","last_undegraded":"2024-02-21T23:53:15.890247+0000","last_fullsized":"2024-02-21T23:53:15.890247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'138","reported_seq":167,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700347+0000","last_change":"2024-02-21T23:25:22.243303+0000","last_active":"2024-02-21T23:52:56.700347+0000","last_peered":"2024-02-21T23:52:56.700347+0000","last_clean":"2024-02-21T23:52:56.700347+0000","last_became_active":"2024-02-21T23:25:22.242960+0000","last_became_peered":"2024-02-21T23:25:22.242960+0000","last_unstale":"2024-02-21T23:52:56.700347+0000","last_undegraded":"2024-02-21T23:52:56.700347+0000","last_fullsized":"2024-02-21T23:52:56.700347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:54:16.386354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":8319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"26'48","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605915+0000","last_change":"2024-02-21T23:25:17.184233+0000","last_active":"2024-02-21T23:53:24.605915+0000","last_peered":"2024-02-21T23:53:24.605915+0000","last_clean":"2024-02-21T23:53:24.605915+0000","last_became_active":"2024-02-21T23:25:17.183980+0000","last_became_peered":"2024-02-21T23:25:17.183980+0000","last_unstale":"2024-02-21T23:53:24.605915+0000","last_undegraded":"2024-02-21T23:53:24.605915+0000","last_fullsized":"2024-02-21T23:53:24.605915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'3","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:14.807121+0000","last_change":"2024-02-21T23:25:16.131147+0000","last_active":"2024-02-21T23:53:14.807121+0000","last_peered":"2024-02-21T23:53:14.807121+0000","last_clean":"2024-02-21T23:53:14.807121+0000","last_became_active":"2024-02-21T23:25:16.130553+0000","last_became_peered":"2024-02-21T23:25:16.130553+0000","last_unstale":"2024-02-21T23:53:14.807121+0000","last_undegraded":"2024-02-21T23:53:14.807121+0000","last_fullsized":"2024-02-21T23:53:14.807121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'97","reported_seq":127,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.566724+0000","last_change":"2024-02-21T23:25:22.243261+0000","last_active":"2024-02-21T23:52:56.566724+0000","last_peered":"2024-02-21T23:52:56.566724+0000","last_clean":"2024-02-21T23:52:56.566724+0000","last_became_active":"2024-02-21T23:25:22.242900+0000","last_became_peered":"2024-02-21T23:25:22.242900+0000","last_unstale":"2024-02-21T23:52:56.566724+0000","last_undegraded":"2024-02-21T23:52:56.566724+0000","last_fullsized":"2024-02-21T23:52:56.566724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:34:44.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":7038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'44","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607089+0000","last_change":"2024-02-21T23:25:17.183303+0000","last_active":"2024-02-21T23:53:24.607089+0000","last_peered":"2024-02-21T23:53:24.607089+0000","last_clean":"2024-02-21T23:53:24.607089+0000","last_became_active":"2024-02-21T23:25:17.183042+0000","last_became_peered":"2024-02-21T23:25:17.183042+0000","last_unstale":"2024-02-21T23:53:24.607089+0000","last_undegraded":"2024-02-21T23:53:24.607089+0000","last_fullsized":"2024-02-21T23:53:24.607089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'251","reported_seq":289,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.291094+0000","last_change":"2024-02-21T23:25:16.127609+0000","last_active":"2024-02-21T23:53:24.291094+0000","last_peered":"2024-02-21T23:53:24.291094+0000","last_clean":"2024-02-21T23:53:24.291094+0000","last_became_active":"2024-02-21T23:25:16.127375+0000","last_became_peered":"2024-02-21T23:25:16.127375+0000","last_unstale":"2024-02-21T23:53:24.291094+0000","last_undegraded":"2024-02-21T23:53:24.291094+0000","last_fullsized":"2024-02-21T23:53:24.291094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2561517,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":2649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'148","reported_seq":177,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.755700+0000","last_change":"2024-02-21T23:25:22.248329+0000","last_active":"2024-02-21T23:52:56.755700+0000","last_peered":"2024-02-21T23:52:56.755700+0000","last_clean":"2024-02-21T23:52:56.755700+0000","last_became_active":"2024-02-21T23:25:22.247889+0000","last_became_peered":"2024-02-21T23:25:22.247889+0000","last_unstale":"2024-02-21T23:52:56.755700+0000","last_undegraded":"2024-02-21T23:52:56.755700+0000","last_fullsized":"2024-02-21T23:52:56.755700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":13204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"26'37","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.527489+0000","last_change":"2024-02-21T23:25:57.527196+0000","last_active":"2024-02-21T23:53:24.527489+0000","last_peered":"2024-02-21T23:53:24.527489+0000","last_clean":"2024-02-21T23:53:24.527489+0000","last_became_active":"2024-02-21T23:25:57.526874+0000","last_became_peered":"2024-02-21T23:25:57.526874+0000","last_unstale":"2024-02-21T23:53:24.527489+0000","last_undegraded":"2024-02-21T23:53:24.527489+0000","last_fullsized":"2024-02-21T23:53:24.527489+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"26'835","reported_seq":873,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:16.019825+0000","last_change":"2024-02-21T23:25:16.116460+0000","last_active":"2024-02-21T23:53:16.019825+0000","last_peered":"2024-02-21T23:53:16.019825+0000","last_clean":"2024-02-21T23:53:16.019825+0000","last_became_active":"2024-02-21T23:25:16.116162+0000","last_became_peered":"2024-02-21T23:25:16.116162+0000","last_unstale":"2024-02-21T23:53:16.019825+0000","last_undegraded":"2024-02-21T23:53:16.019825+0000","last_fullsized":"2024-02-21T23:53:16.019825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'155","reported_seq":185,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.702447+0000","last_change":"2024-02-21T23:25:21.683199+0000","last_active":"2024-02-21T23:52:56.702447+0000","last_peered":"2024-02-21T23:52:56.702447+0000","last_clean":"2024-02-21T23:52:56.702447+0000","last_became_active":"2024-02-21T23:25:21.682425+0000","last_became_peered":"2024-02-21T23:25:21.682425+0000","last_unstale":"2024-02-21T23:52:56.702447+0000","last_undegraded":"2024-02-21T23:52:56.702447+0000","last_fullsized":"2024-02-21T23:52:56.702447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":9421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'32","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.487783+0000","last_change":"2024-02-21T23:25:17.184837+0000","last_active":"2024-02-21T23:53:24.487783+0000","last_peered":"2024-02-21T23:53:24.487783+0000","last_clean":"2024-02-21T23:53:24.487783+0000","last_became_active":"2024-02-21T23:25:17.184574+0000","last_became_peered":"2024-02-21T23:25:17.184574+0000","last_unstale":"2024-02-21T23:53:24.487783+0000","last_undegraded":"2024-02-21T23:53:24.487783+0000","last_fullsized":"2024-02-21T23:53:24.487783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'697","reported_seq":734,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.851348+0000","last_change":"2024-02-21T23:25:16.151031+0000","last_active":"2024-02-21T23:53:18.851348+0000","last_peered":"2024-02-21T23:53:18.851348+0000","last_clean":"2024-02-21T23:53:18.851348+0000","last_became_active":"2024-02-21T23:25:16.150767+0000","last_became_peered":"2024-02-21T23:25:16.150767+0000","last_unstale":"2024-02-21T23:53:18.851348+0000","last_undegraded":"2024-02-21T23:53:18.851348+0000","last_fullsized":"2024-02-21T23:53:18.851348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'152","reported_seq":182,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.422949+0000","last_change":"2024-02-21T23:25:21.683995+0000","last_active":"2024-02-21T23:52:56.422949+0000","last_peered":"2024-02-21T23:52:56.422949+0000","last_clean":"2024-02-21T23:52:56.422949+0000","last_became_active":"2024-02-21T23:25:21.683415+0000","last_became_peered":"2024-02-21T23:25:21.683415+0000","last_unstale":"2024-02-21T23:52:56.422949+0000","last_undegraded":"2024-02-21T23:52:56.422949+0000","last_fullsized":"2024-02-21T23:52:56.422949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"26'42","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604682+0000","last_change":"2024-02-21T23:25:17.187230+0000","last_active":"2024-02-21T23:53:24.604682+0000","last_peered":"2024-02-21T23:53:24.604682+0000","last_clean":"2024-02-21T23:53:24.604682+0000","last_became_active":"2024-02-21T23:25:17.186952+0000","last_became_peered":"2024-02-21T23:25:17.186952+0000","last_unstale":"2024-02-21T23:53:24.604682+0000","last_undegraded":"2024-02-21T23:53:24.604682+0000","last_fullsized":"2024-02-21T23:53:24.604682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:00.113126+0000","last_change":"2024-02-21T23:25:16.134722+0000","last_active":"2024-02-21T23:53:00.113126+0000","last_peered":"2024-02-21T23:53:00.113126+0000","last_clean":"2024-02-21T23:53:00.113126+0000","last_became_active":"2024-02-21T23:25:16.134458+0000","last_became_peered":"2024-02-21T23:25:16.134458+0000","last_unstale":"2024-02-21T23:53:00.113126+0000","last_undegraded":"2024-02-21T23:53:00.113126+0000","last_fullsized":"2024-02-21T23:53:00.113126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'139","reported_seq":169,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700599+0000","last_change":"2024-02-21T23:25:21.684109+0000","last_active":"2024-02-21T23:52:56.700599+0000","last_peered":"2024-02-21T23:52:56.700599+0000","last_clean":"2024-02-21T23:52:56.700599+0000","last_became_active":"2024-02-21T23:25:21.683705+0000","last_became_peered":"2024-02-21T23:25:21.683705+0000","last_unstale":"2024-02-21T23:52:56.700599+0000","last_undegraded":"2024-02-21T23:52:56.700599+0000","last_fullsized":"2024-02-21T23:52:56.700599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":10133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"26'42","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493527+0000","last_change":"2024-02-21T23:25:17.188374+0000","last_active":"2024-02-21T23:53:24.493527+0000","last_peered":"2024-02-21T23:53:24.493527+0000","last_clean":"2024-02-21T23:53:24.493527+0000","last_became_active":"2024-02-21T23:25:17.187792+0000","last_became_peered":"2024-02-21T23:25:17.187792+0000","last_unstale":"2024-02-21T23:53:24.493527+0000","last_undegraded":"2024-02-21T23:53:24.493527+0000","last_fullsized":"2024-02-21T23:53:24.493527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'5","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:01.175408+0000","last_change":"2024-02-21T23:25:16.129446+0000","last_active":"2024-02-21T23:53:01.175408+0000","last_peered":"2024-02-21T23:53:01.175408+0000","last_clean":"2024-02-21T23:53:01.175408+0000","last_became_active":"2024-02-21T23:25:16.129164+0000","last_became_peered":"2024-02-21T23:25:16.129164+0000","last_unstale":"2024-02-21T23:53:01.175408+0000","last_undegraded":"2024-02-21T23:53:01.175408+0000","last_fullsized":"2024-02-21T23:53:01.175408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'139","reported_seq":162,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.403580+0000","last_change":"2024-02-21T23:25:57.600307+0000","last_active":"2024-02-21T23:52:56.403580+0000","last_peered":"2024-02-21T23:52:56.403580+0000","last_clean":"2024-02-21T23:52:56.403580+0000","last_became_active":"2024-02-21T23:25:57.599867+0000","last_became_peered":"2024-02-21T23:25:57.599867+0000","last_unstale":"2024-02-21T23:52:56.403580+0000","last_undegraded":"2024-02-21T23:52:56.403580+0000","last_fullsized":"2024-02-21T23:52:56.403580+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":5393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,5],"acting":[2,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"26'153","reported_seq":185,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.092755+0000","last_change":"2024-02-21T23:25:22.248119+0000","last_active":"2024-02-21T23:52:55.092755+0000","last_peered":"2024-02-21T23:52:55.092755+0000","last_clean":"2024-02-21T23:52:55.092755+0000","last_became_active":"2024-02-21T23:25:22.247811+0000","last_became_peered":"2024-02-21T23:25:22.247811+0000","last_unstale":"2024-02-21T23:52:55.092755+0000","last_undegraded":"2024-02-21T23:52:55.092755+0000","last_fullsized":"2024-02-21T23:52:55.092755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":11393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'41","reported_seq":76,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.466267+0000","last_change":"2024-02-21T23:25:17.189176+0000","last_active":"2024-02-21T23:53:24.466267+0000","last_peered":"2024-02-21T23:53:24.466267+0000","last_clean":"2024-02-21T23:53:24.466267+0000","last_became_active":"2024-02-21T23:25:17.188885+0000","last_became_peered":"2024-02-21T23:25:17.188885+0000","last_unstale":"2024-02-21T23:53:24.466267+0000","last_undegraded":"2024-02-21T23:53:24.466267+0000","last_fullsized":"2024-02-21T23:53:24.466267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'4","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:04.260921+0000","last_change":"2024-02-21T23:25:16.132036+0000","last_active":"2024-02-21T23:53:04.260921+0000","last_peered":"2024-02-21T23:53:04.260921+0000","last_clean":"2024-02-21T23:53:04.260921+0000","last_became_active":"2024-02-21T23:25:16.131756+0000","last_became_peered":"2024-02-21T23:25:16.131756+0000","last_unstale":"2024-02-21T23:53:04.260921+0000","last_undegraded":"2024-02-21T23:53:04.260921+0000","last_fullsized":"2024-02-21T23:53:04.260921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"26'157","reported_seq":188,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.421511+0000","last_change":"2024-02-21T23:25:21.686133+0000","last_active":"2024-02-21T23:52:56.421511+0000","last_peered":"2024-02-21T23:52:56.421511+0000","last_clean":"2024-02-21T23:52:56.421511+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:52:56.421511+0000","last_undegraded":"2024-02-21T23:52:56.421511+0000","last_fullsized":"2024-02-21T23:52:56.421511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":15074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'44","reported_seq":79,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471213+0000","last_change":"2024-02-21T23:25:17.135158+0000","last_active":"2024-02-21T23:53:24.471213+0000","last_peered":"2024-02-21T23:53:24.471213+0000","last_clean":"2024-02-21T23:53:24.471213+0000","last_became_active":"2024-02-21T23:25:17.134888+0000","last_became_peered":"2024-02-21T23:25:17.134888+0000","last_unstale":"2024-02-21T23:53:24.471213+0000","last_undegraded":"2024-02-21T23:53:24.471213+0000","last_fullsized":"2024-02-21T23:53:24.471213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'5","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:10.598178+0000","last_change":"2024-02-21T23:25:16.116378+0000","last_active":"2024-02-21T23:53:10.598178+0000","last_peered":"2024-02-21T23:53:10.598178+0000","last_clean":"2024-02-21T23:53:10.598178+0000","last_became_active":"2024-02-21T23:25:16.116201+0000","last_became_peered":"2024-02-21T23:25:16.116201+0000","last_unstale":"2024-02-21T23:53:10.598178+0000","last_undegraded":"2024-02-21T23:53:10.598178+0000","last_fullsized":"2024-02-21T23:53:10.598178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'718","reported_seq":755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.456812+0000","last_change":"2024-02-21T23:25:16.132314+0000","last_active":"2024-02-21T23:52:48.456812+0000","last_peered":"2024-02-21T23:52:48.456812+0000","last_clean":"2024-02-21T23:52:48.456812+0000","last_became_active":"2024-02-21T23:25:16.132062+0000","last_became_peered":"2024-02-21T23:25:16.132062+0000","last_unstale":"2024-02-21T23:52:48.456812+0000","last_undegraded":"2024-02-21T23:52:48.456812+0000","last_fullsized":"2024-02-21T23:52:48.456812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":716,"num_write_kb":4453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'39","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607247+0000","last_change":"2024-02-21T23:25:17.161451+0000","last_active":"2024-02-21T23:53:24.607247+0000","last_peered":"2024-02-21T23:53:24.607247+0000","last_clean":"2024-02-21T23:53:24.607247+0000","last_became_active":"2024-02-21T23:25:17.161176+0000","last_became_peered":"2024-02-21T23:25:17.161176+0000","last_unstale":"2024-02-21T23:53:24.607247+0000","last_undegraded":"2024-02-21T23:53:24.607247+0000","last_fullsized":"2024-02-21T23:53:24.607247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'95","reported_seq":124,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:53.621659+0000","last_change":"2024-02-21T23:25:21.685724+0000","last_active":"2024-02-21T23:52:53.621659+0000","last_peered":"2024-02-21T23:52:53.621659+0000","last_clean":"2024-02-21T23:52:53.621659+0000","last_became_active":"2024-02-21T23:25:21.685375+0000","last_became_peered":"2024-02-21T23:25:21.685375+0000","last_unstale":"2024-02-21T23:52:53.621659+0000","last_undegraded":"2024-02-21T23:52:53.621659+0000","last_fullsized":"2024-02-21T23:52:53.621659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:56:28.795401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":7997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"26'688","reported_seq":725,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:06.343128+0000","last_change":"2024-02-21T23:25:16.134127+0000","last_active":"2024-02-21T23:53:06.343128+0000","last_peered":"2024-02-21T23:53:06.343128+0000","last_clean":"2024-02-21T23:53:06.343128+0000","last_became_active":"2024-02-21T23:25:16.133839+0000","last_became_peered":"2024-02-21T23:25:16.133839+0000","last_unstale":"2024-02-21T23:53:06.343128+0000","last_undegraded":"2024-02-21T23:53:06.343128+0000","last_fullsized":"2024-02-21T23:53:06.343128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'33","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.483697+0000","last_change":"2024-02-21T23:25:17.190455+0000","last_active":"2024-02-21T23:53:24.483697+0000","last_peered":"2024-02-21T23:53:24.483697+0000","last_clean":"2024-02-21T23:53:24.483697+0000","last_became_active":"2024-02-21T23:25:17.190194+0000","last_became_peered":"2024-02-21T23:25:17.190194+0000","last_unstale":"2024-02-21T23:53:24.483697+0000","last_undegraded":"2024-02-21T23:53:24.483697+0000","last_fullsized":"2024-02-21T23:53:24.483697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:03.451409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'159","reported_seq":195,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.988198+0000","last_change":"2024-02-21T23:25:57.526175+0000","last_active":"2024-02-21T23:52:52.988198+0000","last_peered":"2024-02-21T23:52:52.988198+0000","last_clean":"2024-02-21T23:52:52.988198+0000","last_became_active":"2024-02-21T23:25:57.525755+0000","last_became_peered":"2024-02-21T23:25:57.525755+0000","last_unstale":"2024-02-21T23:52:52.988198+0000","last_undegraded":"2024-02-21T23:52:52.988198+0000","last_fullsized":"2024-02-21T23:52:52.988198+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:18:17.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4,2],"acting":[3,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'48","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626470+0000","last_change":"2024-02-21T23:25:17.138039+0000","last_active":"2024-02-21T23:53:24.626470+0000","last_peered":"2024-02-21T23:53:24.626470+0000","last_clean":"2024-02-21T23:53:24.626470+0000","last_became_active":"2024-02-21T23:25:17.137717+0000","last_became_peered":"2024-02-21T23:25:17.137717+0000","last_unstale":"2024-02-21T23:53:24.626470+0000","last_undegraded":"2024-02-21T23:53:24.626470+0000","last_fullsized":"2024-02-21T23:53:24.626470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"26'48","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.529559+0000","last_change":"2024-02-21T23:25:17.194727+0000","last_active":"2024-02-21T23:53:24.529559+0000","last_peered":"2024-02-21T23:53:24.529559+0000","last_clean":"2024-02-21T23:53:24.529559+0000","last_became_active":"2024-02-21T23:25:17.194533+0000","last_became_peered":"2024-02-21T23:25:17.194533+0000","last_unstale":"2024-02-21T23:53:24.529559+0000","last_undegraded":"2024-02-21T23:53:24.529559+0000","last_fullsized":"2024-02-21T23:53:24.529559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"26'50","reported_seq":87,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493574+0000","last_change":"2024-02-21T23:25:17.191051+0000","last_active":"2024-02-21T23:53:24.493574+0000","last_peered":"2024-02-21T23:53:24.493574+0000","last_clean":"2024-02-21T23:53:24.493574+0000","last_became_active":"2024-02-21T23:25:17.190600+0000","last_became_peered":"2024-02-21T23:25:17.190600+0000","last_unstale":"2024-02-21T23:53:24.493574+0000","last_undegraded":"2024-02-21T23:53:24.493574+0000","last_fullsized":"2024-02-21T23:53:24.493574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.599153+0000","last_change":"2024-02-21T23:25:17.135311+0000","last_active":"2024-02-21T23:53:24.599153+0000","last_peered":"2024-02-21T23:53:24.599153+0000","last_clean":"2024-02-21T23:53:24.599153+0000","last_became_active":"2024-02-21T23:25:17.135011+0000","last_became_peered":"2024-02-21T23:25:17.135011+0000","last_unstale":"2024-02-21T23:53:24.599153+0000","last_undegraded":"2024-02-21T23:53:24.599153+0000","last_fullsized":"2024-02-21T23:53:24.599153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.472250+0000","last_change":"2024-02-21T23:25:17.161696+0000","last_active":"2024-02-21T23:53:24.472250+0000","last_peered":"2024-02-21T23:53:24.472250+0000","last_clean":"2024-02-21T23:53:24.472250+0000","last_became_active":"2024-02-21T23:25:17.161401+0000","last_became_peered":"2024-02-21T23:25:17.161401+0000","last_unstale":"2024-02-21T23:53:24.472250+0000","last_undegraded":"2024-02-21T23:53:24.472250+0000","last_fullsized":"2024-02-21T23:53:24.472250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"26'34","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.463178+0000","last_change":"2024-02-21T23:25:17.189731+0000","last_active":"2024-02-21T23:53:24.463178+0000","last_peered":"2024-02-21T23:53:24.463178+0000","last_clean":"2024-02-21T23:53:24.463178+0000","last_became_active":"2024-02-21T23:25:17.189484+0000","last_became_peered":"2024-02-21T23:25:17.189484+0000","last_unstale":"2024-02-21T23:53:24.463178+0000","last_undegraded":"2024-02-21T23:53:24.463178+0000","last_fullsized":"2024-02-21T23:53:24.463178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"26'45","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.628365+0000","last_change":"2024-02-21T23:25:17.184125+0000","last_active":"2024-02-21T23:53:24.628365+0000","last_peered":"2024-02-21T23:53:24.628365+0000","last_clean":"2024-02-21T23:53:24.628365+0000","last_became_active":"2024-02-21T23:25:17.183865+0000","last_became_peered":"2024-02-21T23:25:17.183865+0000","last_unstale":"2024-02-21T23:53:24.628365+0000","last_undegraded":"2024-02-21T23:53:24.628365+0000","last_fullsized":"2024-02-21T23:53:24.628365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'37","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.468066+0000","last_change":"2024-02-21T23:25:17.161482+0000","last_active":"2024-02-21T23:53:24.468066+0000","last_peered":"2024-02-21T23:53:24.468066+0000","last_clean":"2024-02-21T23:53:24.468066+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:24.468066+0000","last_undegraded":"2024-02-21T23:53:24.468066+0000","last_fullsized":"2024-02-21T23:53:24.468066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'32","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494217+0000","last_change":"2024-02-21T23:25:17.185691+0000","last_active":"2024-02-21T23:53:24.494217+0000","last_peered":"2024-02-21T23:53:24.494217+0000","last_clean":"2024-02-21T23:53:24.494217+0000","last_became_active":"2024-02-21T23:25:17.185420+0000","last_became_peered":"2024-02-21T23:25:17.185420+0000","last_unstale":"2024-02-21T23:53:24.494217+0000","last_undegraded":"2024-02-21T23:53:24.494217+0000","last_fullsized":"2024-02-21T23:53:24.494217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"26'46","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471138+0000","last_change":"2024-02-21T23:25:17.161321+0000","last_active":"2024-02-21T23:53:24.471138+0000","last_peered":"2024-02-21T23:53:24.471138+0000","last_clean":"2024-02-21T23:53:24.471138+0000","last_became_active":"2024-02-21T23:25:17.161010+0000","last_became_peered":"2024-02-21T23:25:17.161010+0000","last_unstale":"2024-02-21T23:53:24.471138+0000","last_undegraded":"2024-02-21T23:53:24.471138+0000","last_fullsized":"2024-02-21T23:53:24.471138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"26'39","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494124+0000","last_change":"2024-02-21T23:25:17.191072+0000","last_active":"2024-02-21T23:53:24.494124+0000","last_peered":"2024-02-21T23:53:24.494124+0000","last_clean":"2024-02-21T23:53:24.494124+0000","last_became_active":"2024-02-21T23:25:17.187741+0000","last_became_peered":"2024-02-21T23:25:17.187741+0000","last_unstale":"2024-02-21T23:53:24.494124+0000","last_undegraded":"2024-02-21T23:53:24.494124+0000","last_fullsized":"2024-02-21T23:53:24.494124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"26'53","reported_seq":89,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607148+0000","last_change":"2024-02-21T23:25:17.186616+0000","last_active":"2024-02-21T23:53:24.607148+0000","last_peered":"2024-02-21T23:53:24.607148+0000","last_clean":"2024-02-21T23:53:24.607148+0000","last_became_active":"2024-02-21T23:25:17.186108+0000","last_became_peered":"2024-02-21T23:25:17.186108+0000","last_unstale":"2024-02-21T23:53:24.607148+0000","last_undegraded":"2024-02-21T23:53:24.607148+0000","last_fullsized":"2024-02-21T23:53:24.607148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"26'43","reported_seq":79,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626297+0000","last_change":"2024-02-21T23:25:17.161428+0000","last_active":"2024-02-21T23:53:24.626297+0000","last_peered":"2024-02-21T23:53:24.626297+0000","last_clean":"2024-02-21T23:53:24.626297+0000","last_became_active":"2024-02-21T23:25:17.161140+0000","last_became_peered":"2024-02-21T23:25:17.161140+0000","last_unstale":"2024-02-21T23:53:24.626297+0000","last_undegraded":"2024-02-21T23:53:24.626297+0000","last_fullsized":"2024-02-21T23:53:24.626297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604720+0000","last_change":"2024-02-21T23:25:17.190666+0000","last_active":"2024-02-21T23:53:24.604720+0000","last_peered":"2024-02-21T23:53:24.604720+0000","last_clean":"2024-02-21T23:53:24.604720+0000","last_became_active":"2024-02-21T23:25:17.190420+0000","last_became_peered":"2024-02-21T23:25:17.190420+0000","last_unstale":"2024-02-21T23:53:24.604720+0000","last_undegraded":"2024-02-21T23:53:24.604720+0000","last_fullsized":"2024-02-21T23:53:24.604720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"26'48","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.531320+0000","last_change":"2024-02-21T23:25:17.189583+0000","last_active":"2024-02-21T23:53:24.531320+0000","last_peered":"2024-02-21T23:53:24.531320+0000","last_clean":"2024-02-21T23:53:24.531320+0000","last_became_active":"2024-02-21T23:25:17.189302+0000","last_became_peered":"2024-02-21T23:25:17.189302+0000","last_unstale":"2024-02-21T23:53:24.531320+0000","last_undegraded":"2024-02-21T23:53:24.531320+0000","last_fullsized":"2024-02-21T23:53:24.531320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"26'41","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493492+0000","last_change":"2024-02-21T23:25:17.188113+0000","last_active":"2024-02-21T23:53:24.493492+0000","last_peered":"2024-02-21T23:53:24.493492+0000","last_clean":"2024-02-21T23:53:24.493492+0000","last_became_active":"2024-02-21T23:25:17.187811+0000","last_became_peered":"2024-02-21T23:25:17.187811+0000","last_unstale":"2024-02-21T23:53:24.493492+0000","last_undegraded":"2024-02-21T23:53:24.493492+0000","last_fullsized":"2024-02-21T23:53:24.493492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"26'44","reported_seq":65,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.530148+0000","last_change":"2024-02-21T23:25:57.525755+0000","last_active":"2024-02-21T23:53:24.530148+0000","last_peered":"2024-02-21T23:53:24.530148+0000","last_clean":"2024-02-21T23:53:24.530148+0000","last_became_active":"2024-02-21T23:25:57.525443+0000","last_became_peered":"2024-02-21T23:25:57.525443+0000","last_unstale":"2024-02-21T23:53:24.530148+0000","last_undegraded":"2024-02-21T23:53:24.530148+0000","last_fullsized":"2024-02-21T23:53:24.530148+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.457278+0000","last_change":"2024-02-21T23:25:17.186378+0000","last_active":"2024-02-21T23:53:24.457278+0000","last_peered":"2024-02-21T23:53:24.457278+0000","last_clean":"2024-02-21T23:53:24.457278+0000","last_became_active":"2024-02-21T23:25:17.186086+0000","last_became_peered":"2024-02-21T23:25:17.186086+0000","last_unstale":"2024-02-21T23:53:24.457278+0000","last_undegraded":"2024-02-21T23:53:24.457278+0000","last_fullsized":"2024-02-21T23:53:24.457278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"26'41","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.602175+0000","last_change":"2024-02-21T23:25:17.190934+0000","last_active":"2024-02-21T23:53:24.602175+0000","last_peered":"2024-02-21T23:53:24.602175+0000","last_clean":"2024-02-21T23:53:24.602175+0000","last_became_active":"2024-02-21T23:25:17.190680+0000","last_became_peered":"2024-02-21T23:25:17.190680+0000","last_unstale":"2024-02-21T23:53:24.602175+0000","last_undegraded":"2024-02-21T23:53:24.602175+0000","last_fullsized":"2024-02-21T23:53:24.602175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:22:04.238374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"26'38","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.473047+0000","last_change":"2024-02-21T23:25:17.135140+0000","last_active":"2024-02-21T23:53:24.473047+0000","last_peered":"2024-02-21T23:53:24.473047+0000","last_clean":"2024-02-21T23:53:24.473047+0000","last_became_active":"2024-02-21T23:25:17.134866+0000","last_became_peered":"2024-02-21T23:25:17.134866+0000","last_unstale":"2024-02-21T23:53:24.473047+0000","last_undegraded":"2024-02-21T23:53:24.473047+0000","last_fullsized":"2024-02-21T23:53:24.473047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'27","reported_seq":62,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.488701+0000","last_change":"2024-02-21T23:25:17.162204+0000","last_active":"2024-02-21T23:53:24.488701+0000","last_peered":"2024-02-21T23:53:24.488701+0000","last_clean":"2024-02-21T23:53:24.488701+0000","last_became_active":"2024-02-21T23:25:17.161807+0000","last_became_peered":"2024-02-21T23:25:17.161807+0000","last_unstale":"2024-02-21T23:53:24.488701+0000","last_undegraded":"2024-02-21T23:53:24.488701+0000","last_fullsized":"2024-02-21T23:53:24.488701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"26'36","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626279+0000","last_change":"2024-02-21T23:25:17.188932+0000","last_active":"2024-02-21T23:53:24.626279+0000","last_peered":"2024-02-21T23:53:24.626279+0000","last_clean":"2024-02-21T23:53:24.626279+0000","last_became_active":"2024-02-21T23:25:17.188646+0000","last_became_peered":"2024-02-21T23:25:17.188646+0000","last_unstale":"2024-02-21T23:53:24.626279+0000","last_undegraded":"2024-02-21T23:53:24.626279+0000","last_fullsized":"2024-02-21T23:53:24.626279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"26'51","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.628333+0000","last_change":"2024-02-21T23:25:17.188333+0000","last_active":"2024-02-21T23:53:24.628333+0000","last_peered":"2024-02-21T23:53:24.628333+0000","last_clean":"2024-02-21T23:53:24.628333+0000","last_became_active":"2024-02-21T23:25:17.188066+0000","last_became_peered":"2024-02-21T23:25:17.188066+0000","last_unstale":"2024-02-21T23:53:24.628333+0000","last_undegraded":"2024-02-21T23:53:24.628333+0000","last_fullsized":"2024-02-21T23:53:24.628333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"26'39","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605416+0000","last_change":"2024-02-21T23:25:17.187549+0000","last_active":"2024-02-21T23:53:24.605416+0000","last_peered":"2024-02-21T23:53:24.605416+0000","last_clean":"2024-02-21T23:53:24.605416+0000","last_became_active":"2024-02-21T23:25:17.187297+0000","last_became_peered":"2024-02-21T23:25:17.187297+0000","last_unstale":"2024-02-21T23:53:24.605416+0000","last_undegraded":"2024-02-21T23:53:24.605416+0000","last_fullsized":"2024-02-21T23:53:24.605416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"26'30","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.535065+0000","last_change":"2024-02-21T23:25:57.526373+0000","last_active":"2024-02-21T23:53:24.535065+0000","last_peered":"2024-02-21T23:53:24.535065+0000","last_clean":"2024-02-21T23:53:24.535065+0000","last_became_active":"2024-02-21T23:25:57.526068+0000","last_became_peered":"2024-02-21T23:25:57.526068+0000","last_unstale":"2024-02-21T23:53:24.535065+0000","last_undegraded":"2024-02-21T23:53:24.535065+0000","last_fullsized":"2024-02-21T23:53:24.535065+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"26'37","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.597841+0000","last_change":"2024-02-21T23:25:17.183264+0000","last_active":"2024-02-21T23:53:24.597841+0000","last_peered":"2024-02-21T23:53:24.597841+0000","last_clean":"2024-02-21T23:53:24.597841+0000","last_became_active":"2024-02-21T23:25:17.182924+0000","last_became_peered":"2024-02-21T23:25:17.182924+0000","last_unstale":"2024-02-21T23:53:24.597841+0000","last_undegraded":"2024-02-21T23:53:24.597841+0000","last_fullsized":"2024-02-21T23:53:24.597841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"26'38","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494000+0000","last_change":"2024-02-21T23:25:17.190841+0000","last_active":"2024-02-21T23:53:24.494000+0000","last_peered":"2024-02-21T23:53:24.494000+0000","last_clean":"2024-02-21T23:53:24.494000+0000","last_became_active":"2024-02-21T23:25:17.190276+0000","last_became_peered":"2024-02-21T23:25:17.190276+0000","last_unstale":"2024-02-21T23:53:24.494000+0000","last_undegraded":"2024-02-21T23:53:24.494000+0000","last_fullsized":"2024-02-21T23:53:24.494000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'36","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.601860+0000","last_change":"2024-02-21T23:25:17.189872+0000","last_active":"2024-02-21T23:53:24.601860+0000","last_peered":"2024-02-21T23:53:24.601860+0000","last_clean":"2024-02-21T23:53:24.601860+0000","last_became_active":"2024-02-21T23:25:17.189614+0000","last_became_peered":"2024-02-21T23:25:17.189614+0000","last_unstale":"2024-02-21T23:53:24.601860+0000","last_undegraded":"2024-02-21T23:53:24.601860+0000","last_fullsized":"2024-02-21T23:53:24.601860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"26'25","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493930+0000","last_change":"2024-02-21T23:25:57.600458+0000","last_active":"2024-02-21T23:53:24.493930+0000","last_peered":"2024-02-21T23:53:24.493930+0000","last_clean":"2024-02-21T23:53:24.493930+0000","last_became_active":"2024-02-21T23:25:57.600094+0000","last_became_peered":"2024-02-21T23:25:57.600094+0000","last_unstale":"2024-02-21T23:53:24.493930+0000","last_undegraded":"2024-02-21T23:53:24.493930+0000","last_fullsized":"2024-02-21T23:53:24.493930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:20:16.177299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"26'44","reported_seq":81,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.602153+0000","last_change":"2024-02-21T23:25:17.191159+0000","last_active":"2024-02-21T23:53:24.602153+0000","last_peered":"2024-02-21T23:53:24.602153+0000","last_clean":"2024-02-21T23:53:24.602153+0000","last_became_active":"2024-02-21T23:25:17.190576+0000","last_became_peered":"2024-02-21T23:25:17.190576+0000","last_unstale":"2024-02-21T23:53:24.602153+0000","last_undegraded":"2024-02-21T23:53:24.602153+0000","last_fullsized":"2024-02-21T23:53:24.602153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"26'34","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.452090+0000","last_change":"2024-02-21T23:25:17.189287+0000","last_active":"2024-02-21T23:53:24.452090+0000","last_peered":"2024-02-21T23:53:24.452090+0000","last_clean":"2024-02-21T23:53:24.452090+0000","last_became_active":"2024-02-21T23:25:17.189029+0000","last_became_peered":"2024-02-21T23:25:17.189029+0000","last_unstale":"2024-02-21T23:53:24.452090+0000","last_undegraded":"2024-02-21T23:53:24.452090+0000","last_fullsized":"2024-02-21T23:53:24.452090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605166+0000","last_change":"2024-02-21T23:25:17.183737+0000","last_active":"2024-02-21T23:53:24.605166+0000","last_peered":"2024-02-21T23:53:24.605166+0000","last_clean":"2024-02-21T23:53:24.605166+0000","last_became_active":"2024-02-21T23:25:17.183410+0000","last_became_peered":"2024-02-21T23:25:17.183410+0000","last_unstale":"2024-02-21T23:53:24.605166+0000","last_undegraded":"2024-02-21T23:53:24.605166+0000","last_fullsized":"2024-02-21T23:53:24.605166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4087,"num_write_kb":323834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4616,"ondisk_log_size":4616,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2419,"ondisk_log_size":2419,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":40532172,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":11557,"num_write_kb":45747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55631872,"data_stored":81064344,"data_compressed":7543912,"data_compressed_allocated":11722752,"data_compressed_original":37216256,"omap_allocated":0,"internal_metadata":0},"log_size":9530,"ondisk_log_size":9530,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575193,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78552,"kb_used_data":16848,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94293288,"statfs":{"total":96636764160,"available":96556326912,"internally_reserved":0,"allocated":17252352,"data_stored":25871404,"data_compressed":2449552,"data_compressed_allocated":3768320,"data_compressed_original":12558336,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575193,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61012,"kb_used_data":11788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310828,"statfs":{"total":96636764160,"available":96574287872,"internally_reserved":0,"allocated":12070912,"data_stored":16892518,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575193,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":114720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575193,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60916,"kb_used_data":7532,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310924,"statfs":{"total":96636764160,"available":96574386176,"internally_reserved":0,"allocated":7712768,"data_stored":11064847,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575193,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58952,"kb_used_data":9728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312888,"statfs":{"total":96636764160,"available":96576397312,"internally_reserved":0,"allocated":9961472,"data_stored":12918697,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575193,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":495616,"data_stored":335113,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575193,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47824,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324016,"statfs":{"total":96636764160,"available":96587792384,"internally_reserved":0,"allocated":2826240,"data_stored":4309094,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575193,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60600,"kb_used_data":7216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311240,"statfs":{"total":96636764160,"available":96574709760,"internally_reserved":0,"allocated":7389184,"data_stored":11655727,"data_compressed":1158786,"data_compressed_allocated":1720320,"data_compressed_original":6168576,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7045120,"data_stored":10950729,"data_compressed":1126387,"data_compressed_allocated":1675264,"data_compressed_original":5595136,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2547712,"data_stored":4194464,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":220483,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9682944,"data_stored":12804067,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7434240,"data_stored":10950217,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11792384,"data_stored":16777888,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16908288,"data_stored":25166406,"data_compressed":2417153,"data_compressed_allocated":3723264,"data_compressed_original":11984896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-21T23:53:31.198 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-21T23:53:31.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.340+0000 7f4bc0836640 1 Processor -- start 2024-02-21T23:53:31.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.340+0000 7f4bc0836640 1 -- start start 2024-02-21T23:53:31.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.341+0000 7f4bc0836640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4bbc0c4fc0 0x7f4bbc071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.341+0000 7f4bc0836640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 0x7f4bbc06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.341+0000 7f4bc0836640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bbc158230 con 0x7f4bbc06f700 2024-02-21T23:53:31.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.341+0000 7f4bc0836640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bbc158eb0 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.341+0000 7f4bc0836640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc158510 con 0x7f4bbc0c53a0 2024-02-21T23:53:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.342+0000 7f4bba59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4bbc0c4fc0 0x7f4bbc071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.342+0000 7f4bbad9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4bbc06f700 0x7f4bbc15b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44462/0 (socket says 172.21.15.133:44462) 2024-02-21T23:53:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.342+0000 7f4bbad9d640 1 -- 172.21.15.133:0/174822543 learned_addr learned my addr 172.21.15.133:0/174822543 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.342+0000 7f4bb9d9b640 1 --2- 172.21.15.133:0/174822543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 0x7f4bbc06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.344+0000 7f4bb959a640 1 -- 172.21.15.133:0/174822543 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638337836 0 0) 0x7f4bbc158230 con 0x7f4bbc06f700 2024-02-21T23:53:31.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.344+0000 7f4bb959a640 1 -- 172.21.15.133:0/174822543 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba4003520 con 0x7f4bbc06f700 2024-02-21T23:53:31.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.344+0000 7f4bba59c640 1 -- 172.21.15.133:0/174822543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 msgr2=0x7f4bbc06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:31.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.344+0000 7f4bba59c640 1 --2- 172.21.15.133:0/174822543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 0x7f4bbc06f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.344+0000 7f4bba59c640 1 -- 172.21.15.133:0/174822543 >> v1:172.21.15.133:6789/0 conn(0x7f4bbc06f700 legacy=0x7f4bbc15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.344+0000 7f4bba59c640 1 -- 172.21.15.133:0/174822543 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc1587f0 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.346+0000 7f4bba59c640 1 --2- 172.21.15.133:0/174822543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 0x7f4bbc071b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40097a0 tx=0x7f4bb402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5772e64f13126022 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:31.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.350+0000 7f4bb959a640 1 -- 172.21.15.133:0/174822543 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4034070 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.350+0000 7f4bb959a640 1 -- 172.21.15.133:0/174822543 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb4030610 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.350+0000 7f4bb959a640 1 -- 172.21.15.133:0/174822543 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40309e0 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.351+0000 7f4bc0836640 1 -- 172.21.15.133:0/174822543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 msgr2=0x7f4bbc071b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.351+0000 7f4bc0836640 1 --2- 172.21.15.133:0/174822543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 0x7f4bbc071b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40097a0 tx=0x7f4bb402b960 comp rx=0 tx=0).stop 2024-02-21T23:53:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.351+0000 7f4bc0836640 1 -- 172.21.15.133:0/174822543 shutdown_connections 2024-02-21T23:53:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.351+0000 7f4bc0836640 1 --2- 172.21.15.133:0/174822543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 0x7f4bbc06f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.351+0000 7f4bc0836640 1 --2- 172.21.15.133:0/174822543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 0x7f4bbc071b60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.351+0000 7f4bc0836640 1 -- 172.21.15.133:0/174822543 >> 172.21.15.133:0/174822543 conn(0x7f4bbc06aec0 msgr2=0x7f4bbc06b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.351+0000 7f4bc0836640 1 -- 172.21.15.133:0/174822543 shutdown_connections 2024-02-21T23:53:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.351+0000 7f4bc0836640 1 -- 172.21.15.133:0/174822543 wait complete. 2024-02-21T23:53:31.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.352+0000 7f4bc0836640 1 Processor -- start 2024-02-21T23:53:31.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.352+0000 7f4bc0836640 1 -- start start 2024-02-21T23:53:31.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.353+0000 7f4bc0836640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bbc06f700 0x7f4bbc1edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.353+0000 7f4bc0836640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 0x7f4bbc0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.353+0000 7f4bc0836640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 0x7f4bbc0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.353+0000 7f4bc0836640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc15bec0 con 0x7f4bbc06f700 2024-02-21T23:53:31.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.353+0000 7f4bc0836640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc157f20 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.353+0000 7f4bc0836640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc15c040 con 0x7f4bbc0c53a0 2024-02-21T23:53:31.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.353+0000 7f4bb9d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 0x7f4bbc0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.353+0000 7f4bba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bbc06f700 0x7f4bbc1edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.353+0000 7f4bbad9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 0x7f4bbc0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.354+0000 7f4bb9d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 0x7f4bbc0c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:55924/0 (socket says 172.21.15.133:55924) 2024-02-21T23:53:31.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.354+0000 7f4bb9d9b640 1 -- 172.21.15.133:0/2903388315 learned_addr learned my addr 172.21.15.133:0/2903388315 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:31.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.357+0000 7f4bb9d9b640 1 -- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 msgr2=0x7f4bbc0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.357+0000 7f4bb9d9b640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 0x7f4bbc0cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.357+0000 7f4bb9d9b640 1 -- 172.21.15.133:0/2903388315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bbc06f700 msgr2=0x7f4bbc1edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.357+0000 7f4bb9d9b640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bbc06f700 0x7f4bbc1edcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.357+0000 7f4bb9d9b640 1 -- 172.21.15.133:0/2903388315 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc1ee210 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.359+0000 7f4bb9d9b640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 0x7f4bbc0c9f60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000e870 tx=0x7f4bb000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:31.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.363+0000 7f4baaffd640 1 -- 172.21.15.133:0/2903388315 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0017070 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.363+0000 7f4baaffd640 1 -- 172.21.15.133:0/2903388315 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb000c940 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.363+0000 7f4baaffd640 1 -- 172.21.15.133:0/2903388315 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000cd30 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.363+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bbc0ca4a0 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.363+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bbc0cc420 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.364+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bbc0c1c50 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.617+0000 7f4baaffd640 1 -- 172.21.15.133:0/2903388315 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bb001a070 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.617+0000 7f4baaffd640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4b9c06b4d0 0x7f4b9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.617+0000 7f4bba59c640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4b9c06b4d0 0x7f4b9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.618+0000 7f4bba59c640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4b9c06b4d0 0x7f4b9c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4036000 tx=0x7f4bb403b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:31.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.623+0000 7f4baaffd640 1 -- 172.21.15.133:0/2903388315 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f4bb0057080 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.633+0000 7f4baaffd640 1 -- 172.21.15.133:0/2903388315 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f4bbc0c1c50 con 0x7f4bbc0c4fc0 2024-02-21T23:53:31.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.805+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bbc0c96b0 con 0x7f4b9c06b4d0 2024-02-21T23:53:31.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.813+0000 7f4baaffd640 1 -- 172.21.15.133:0/2903388315 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+333996 (secure 0 0 0) 0x7f4bbc0c96b0 con 0x7f4b9c06b4d0 2024-02-21T23:53:31.812 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:53:31.813 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-02-21T23:53:31.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4b9c06b4d0 msgr2=0x7f4b9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4b9c06b4d0 0x7f4b9c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4036000 tx=0x7f4bb403b040 comp rx=0 tx=0).stop 2024-02-21T23:53:31.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 msgr2=0x7f4bbc0c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 0x7f4bbc0c9f60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000e870 tx=0x7f4bb000ed30 comp rx=0 tx=0).stop 2024-02-21T23:53:31.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 shutdown_connections 2024-02-21T23:53:31.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f4b9c06b4d0 0x7f4b9c06d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bbc0c53a0 0x7f4bbc0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bbc0c4fc0 0x7f4bbc0c9f60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 --2- 172.21.15.133:0/2903388315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bbc06f700 0x7f4bbc1edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 >> 172.21.15.133:0/2903388315 conn(0x7f4bbc06aec0 msgr2=0x7f4bbc06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:31.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 shutdown_connections 2024-02-21T23:53:31.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.816+0000 7f4bc0836640 1 -- 172.21.15.133:0/2903388315 wait complete. 2024-02-21T23:53:31.823 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":873,"stamp":"2024-02-21T23:53:31.042599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":41122540,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":1110,"num_read_kb":1160,"num_write":15803,"num_write_kb":371997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16682,"ondisk_log_size":16682,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":426808,"kb_used_data":56632,"kb_used_omap":12,"kb_used_meta":370099,"kb_avail":754547912,"statfs":{"total":773094113280,"available":772657061888,"internally_reserved":0,"allocated":57991168,"data_stored":83162120,"data_compressed":7608710,"data_compressed_allocated":11812864,"data_compressed_original":38363136,"omap_allocated":12750,"internal_metadata":378981938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14811,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":15,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002580"},"pg_stats":[{"pgid":"2.1f","version":"26'604","reported_seq":643,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:17.860114+0000","last_change":"2024-02-21T23:25:16.134390+0000","last_active":"2024-02-21T23:53:17.860114+0000","last_peered":"2024-02-21T23:53:17.860114+0000","last_clean":"2024-02-21T23:53:17.860114+0000","last_became_active":"2024-02-21T23:25:16.134112+0000","last_became_peered":"2024-02-21T23:25:16.134112+0000","last_unstale":"2024-02-21T23:53:17.860114+0000","last_undegraded":"2024-02-21T23:53:17.860114+0000","last_fullsized":"2024-02-21T23:53:17.860114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'30","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.462957+0000","last_change":"2024-02-21T23:25:17.135738+0000","last_active":"2024-02-21T23:53:24.462957+0000","last_peered":"2024-02-21T23:53:24.462957+0000","last_clean":"2024-02-21T23:53:24.462957+0000","last_became_active":"2024-02-21T23:25:17.135586+0000","last_became_peered":"2024-02-21T23:25:17.135586+0000","last_unstale":"2024-02-21T23:53:24.462957+0000","last_undegraded":"2024-02-21T23:53:24.462957+0000","last_fullsized":"2024-02-21T23:53:24.462957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'161","reported_seq":190,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.384215+0000","last_change":"2024-02-21T23:25:22.248962+0000","last_active":"2024-02-21T23:52:56.384215+0000","last_peered":"2024-02-21T23:52:56.384215+0000","last_clean":"2024-02-21T23:52:56.384215+0000","last_became_active":"2024-02-21T23:25:22.248610+0000","last_became_peered":"2024-02-21T23:25:22.248610+0000","last_unstale":"2024-02-21T23:52:56.384215+0000","last_undegraded":"2024-02-21T23:52:56.384215+0000","last_fullsized":"2024-02-21T23:52:56.384215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:45.939036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":10997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'750","reported_seq":790,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:05.342762+0000","last_change":"2024-02-21T23:25:16.123635+0000","last_active":"2024-02-21T23:53:05.342762+0000","last_peered":"2024-02-21T23:53:05.342762+0000","last_clean":"2024-02-21T23:53:05.342762+0000","last_became_active":"2024-02-21T23:25:16.123374+0000","last_became_peered":"2024-02-21T23:25:16.123374+0000","last_unstale":"2024-02-21T23:53:05.342762+0000","last_undegraded":"2024-02-21T23:53:05.342762+0000","last_fullsized":"2024-02-21T23:53:05.342762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":4534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604770+0000","last_change":"2024-02-21T23:25:17.192858+0000","last_active":"2024-02-21T23:53:24.604770+0000","last_peered":"2024-02-21T23:53:24.604770+0000","last_clean":"2024-02-21T23:53:24.604770+0000","last_became_active":"2024-02-21T23:25:17.192626+0000","last_became_peered":"2024-02-21T23:25:17.192626+0000","last_unstale":"2024-02-21T23:53:24.604770+0000","last_undegraded":"2024-02-21T23:53:24.604770+0000","last_fullsized":"2024-02-21T23:53:24.604770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'132","reported_seq":168,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.423235+0000","last_change":"2024-02-21T23:25:57.583837+0000","last_active":"2024-02-21T23:52:56.423235+0000","last_peered":"2024-02-21T23:52:56.423235+0000","last_clean":"2024-02-21T23:52:56.423235+0000","last_became_active":"2024-02-21T23:25:57.583416+0000","last_became_peered":"2024-02-21T23:25:57.583416+0000","last_unstale":"2024-02-21T23:52:56.423235+0000","last_undegraded":"2024-02-21T23:52:56.423235+0000","last_fullsized":"2024-02-21T23:52:56.423235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:25:24.780185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":6267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"26'1343","reported_seq":1380,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.892783+0000","last_change":"2024-02-21T23:25:16.134805+0000","last_active":"2024-02-21T23:53:15.892783+0000","last_peered":"2024-02-21T23:53:15.892783+0000","last_clean":"2024-02-21T23:53:15.892783+0000","last_became_active":"2024-02-21T23:25:16.134533+0000","last_became_peered":"2024-02-21T23:25:16.134533+0000","last_unstale":"2024-02-21T23:53:15.892783+0000","last_undegraded":"2024-02-21T23:53:15.892783+0000","last_fullsized":"2024-02-21T23:53:15.892783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1334,"num_write_kb":8974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'32","reported_seq":67,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494748+0000","last_change":"2024-02-21T23:25:17.182131+0000","last_active":"2024-02-21T23:53:24.494748+0000","last_peered":"2024-02-21T23:53:24.494748+0000","last_clean":"2024-02-21T23:53:24.494748+0000","last_became_active":"2024-02-21T23:25:17.181886+0000","last_became_peered":"2024-02-21T23:25:17.181886+0000","last_unstale":"2024-02-21T23:53:24.494748+0000","last_undegraded":"2024-02-21T23:53:24.494748+0000","last_fullsized":"2024-02-21T23:53:24.494748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'141","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.423026+0000","last_change":"2024-02-21T23:25:21.683506+0000","last_active":"2024-02-21T23:52:56.423026+0000","last_peered":"2024-02-21T23:52:56.423026+0000","last_clean":"2024-02-21T23:52:56.423026+0000","last_became_active":"2024-02-21T23:25:21.683115+0000","last_became_peered":"2024-02-21T23:25:21.683115+0000","last_unstale":"2024-02-21T23:52:56.423026+0000","last_undegraded":"2024-02-21T23:52:56.423026+0000","last_fullsized":"2024-02-21T23:52:56.423026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:13.865145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":12261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"26'6","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:08.503461+0000","last_change":"2024-02-21T23:25:16.130851+0000","last_active":"2024-02-21T23:53:08.503461+0000","last_peered":"2024-02-21T23:53:08.503461+0000","last_clean":"2024-02-21T23:53:08.503461+0000","last_became_active":"2024-02-21T23:25:16.130604+0000","last_became_peered":"2024-02-21T23:25:16.130604+0000","last_unstale":"2024-02-21T23:53:08.503461+0000","last_undegraded":"2024-02-21T23:53:08.503461+0000","last_fullsized":"2024-02-21T23:53:08.503461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'47","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.495013+0000","last_change":"2024-02-21T23:25:17.192642+0000","last_active":"2024-02-21T23:53:24.495013+0000","last_peered":"2024-02-21T23:53:24.495013+0000","last_clean":"2024-02-21T23:53:24.495013+0000","last_became_active":"2024-02-21T23:25:17.192394+0000","last_became_peered":"2024-02-21T23:25:17.192394+0000","last_unstale":"2024-02-21T23:53:24.495013+0000","last_undegraded":"2024-02-21T23:53:24.495013+0000","last_fullsized":"2024-02-21T23:53:24.495013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'145","reported_seq":174,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:53.583749+0000","last_change":"2024-02-21T23:25:21.679491+0000","last_active":"2024-02-21T23:52:53.583749+0000","last_peered":"2024-02-21T23:52:53.583749+0000","last_clean":"2024-02-21T23:52:53.583749+0000","last_became_active":"2024-02-21T23:25:21.679117+0000","last_became_peered":"2024-02-21T23:25:21.679117+0000","last_unstale":"2024-02-21T23:52:53.583749+0000","last_undegraded":"2024-02-21T23:52:53.583749+0000","last_fullsized":"2024-02-21T23:52:53.583749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:52.680971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":9573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626233+0000","last_change":"2024-02-21T23:25:17.161438+0000","last_active":"2024-02-21T23:53:24.626233+0000","last_peered":"2024-02-21T23:53:24.626233+0000","last_clean":"2024-02-21T23:53:24.626233+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:24.626233+0000","last_undegraded":"2024-02-21T23:53:24.626233+0000","last_fullsized":"2024-02-21T23:53:24.626233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'648","reported_seq":686,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:07.478231+0000","last_change":"2024-02-21T23:25:16.136913+0000","last_active":"2024-02-21T23:53:07.478231+0000","last_peered":"2024-02-21T23:53:07.478231+0000","last_clean":"2024-02-21T23:53:07.478231+0000","last_became_active":"2024-02-21T23:25:16.136628+0000","last_became_peered":"2024-02-21T23:25:16.136628+0000","last_unstale":"2024-02-21T23:53:07.478231+0000","last_undegraded":"2024-02-21T23:53:07.478231+0000","last_fullsized":"2024-02-21T23:53:07.478231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":644,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'134","reported_seq":171,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.695314+0000","last_change":"2024-02-21T23:25:57.526324+0000","last_active":"2024-02-21T23:52:56.695314+0000","last_peered":"2024-02-21T23:52:56.695314+0000","last_clean":"2024-02-21T23:52:56.695314+0000","last_became_active":"2024-02-21T23:25:57.525854+0000","last_became_peered":"2024-02-21T23:25:57.525854+0000","last_unstale":"2024-02-21T23:52:56.695314+0000","last_undegraded":"2024-02-21T23:52:56.695314+0000","last_fullsized":"2024-02-21T23:52:56.695314+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:12:32.356243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.465980+0000","last_change":"2024-02-21T23:25:17.137829+0000","last_active":"2024-02-21T23:53:24.465980+0000","last_peered":"2024-02-21T23:53:24.465980+0000","last_clean":"2024-02-21T23:53:24.465980+0000","last_became_active":"2024-02-21T23:25:17.137557+0000","last_became_peered":"2024-02-21T23:25:17.137557+0000","last_unstale":"2024-02-21T23:53:24.465980+0000","last_undegraded":"2024-02-21T23:53:24.465980+0000","last_fullsized":"2024-02-21T23:53:24.465980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:59.099888+0000","last_change":"2024-02-21T23:25:16.146181+0000","last_active":"2024-02-21T23:52:59.099888+0000","last_peered":"2024-02-21T23:52:59.099888+0000","last_clean":"2024-02-21T23:52:59.099888+0000","last_became_active":"2024-02-21T23:25:16.145932+0000","last_became_peered":"2024-02-21T23:25:16.145932+0000","last_unstale":"2024-02-21T23:52:59.099888+0000","last_undegraded":"2024-02-21T23:52:59.099888+0000","last_fullsized":"2024-02-21T23:52:59.099888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'165","reported_seq":195,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.095405+0000","last_change":"2024-02-21T23:25:21.686044+0000","last_active":"2024-02-21T23:52:55.095405+0000","last_peered":"2024-02-21T23:52:55.095405+0000","last_clean":"2024-02-21T23:52:55.095405+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:52:55.095405+0000","last_undegraded":"2024-02-21T23:52:55.095405+0000","last_fullsized":"2024-02-21T23:52:55.095405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:39.740454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'37","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471561+0000","last_change":"2024-02-21T23:25:17.190236+0000","last_active":"2024-02-21T23:53:24.471561+0000","last_peered":"2024-02-21T23:53:24.471561+0000","last_clean":"2024-02-21T23:53:24.471561+0000","last_became_active":"2024-02-21T23:25:17.189957+0000","last_became_peered":"2024-02-21T23:25:17.189957+0000","last_unstale":"2024-02-21T23:53:24.471561+0000","last_undegraded":"2024-02-21T23:53:24.471561+0000","last_fullsized":"2024-02-21T23:53:24.471561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:03.326866+0000","last_change":"2024-02-21T23:25:16.148108+0000","last_active":"2024-02-21T23:53:03.326866+0000","last_peered":"2024-02-21T23:53:03.326866+0000","last_clean":"2024-02-21T23:53:03.326866+0000","last_became_active":"2024-02-21T23:25:16.147930+0000","last_became_peered":"2024-02-21T23:25:16.147930+0000","last_unstale":"2024-02-21T23:53:03.326866+0000","last_undegraded":"2024-02-21T23:53:03.326866+0000","last_fullsized":"2024-02-21T23:53:03.326866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'132","reported_seq":162,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.410926+0000","last_change":"2024-02-21T23:25:22.248150+0000","last_active":"2024-02-21T23:52:56.410926+0000","last_peered":"2024-02-21T23:52:56.410926+0000","last_clean":"2024-02-21T23:52:56.410926+0000","last_became_active":"2024-02-21T23:25:22.247769+0000","last_became_peered":"2024-02-21T23:25:22.247769+0000","last_unstale":"2024-02-21T23:52:56.410926+0000","last_undegraded":"2024-02-21T23:52:56.410926+0000","last_fullsized":"2024-02-21T23:52:56.410926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"26'34","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.477731+0000","last_change":"2024-02-21T23:25:17.189758+0000","last_active":"2024-02-21T23:53:24.477731+0000","last_peered":"2024-02-21T23:53:24.477731+0000","last_clean":"2024-02-21T23:53:24.477731+0000","last_became_active":"2024-02-21T23:25:17.189492+0000","last_became_peered":"2024-02-21T23:25:17.189492+0000","last_unstale":"2024-02-21T23:53:24.477731+0000","last_undegraded":"2024-02-21T23:53:24.477731+0000","last_fullsized":"2024-02-21T23:53:24.477731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:42.526598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'7","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:17.858179+0000","last_change":"2024-02-21T23:25:16.131611+0000","last_active":"2024-02-21T23:53:17.858179+0000","last_peered":"2024-02-21T23:53:17.858179+0000","last_clean":"2024-02-21T23:53:17.858179+0000","last_became_active":"2024-02-21T23:25:16.131158+0000","last_became_peered":"2024-02-21T23:25:16.131158+0000","last_unstale":"2024-02-21T23:53:17.858179+0000","last_undegraded":"2024-02-21T23:53:17.858179+0000","last_fullsized":"2024-02-21T23:53:17.858179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'116","reported_seq":146,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.014642+0000","last_change":"2024-02-21T23:25:21.683821+0000","last_active":"2024-02-21T23:52:52.014642+0000","last_peered":"2024-02-21T23:52:52.014642+0000","last_clean":"2024-02-21T23:52:52.014642+0000","last_became_active":"2024-02-21T23:25:21.683466+0000","last_became_peered":"2024-02-21T23:25:21.683466+0000","last_unstale":"2024-02-21T23:52:52.014642+0000","last_undegraded":"2024-02-21T23:52:52.014642+0000","last_fullsized":"2024-02-21T23:52:52.014642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":7338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"26'38","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.602687+0000","last_change":"2024-02-21T23:25:17.191394+0000","last_active":"2024-02-21T23:53:24.602687+0000","last_peered":"2024-02-21T23:53:24.602687+0000","last_clean":"2024-02-21T23:53:24.602687+0000","last_became_active":"2024-02-21T23:25:17.191106+0000","last_became_peered":"2024-02-21T23:25:17.191106+0000","last_unstale":"2024-02-21T23:53:24.602687+0000","last_undegraded":"2024-02-21T23:53:24.602687+0000","last_fullsized":"2024-02-21T23:53:24.602687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'9","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.843551+0000","last_change":"2024-02-21T23:25:16.134862+0000","last_active":"2024-02-21T23:53:18.843551+0000","last_peered":"2024-02-21T23:53:18.843551+0000","last_clean":"2024-02-21T23:53:18.843551+0000","last_became_active":"2024-02-21T23:25:16.134602+0000","last_became_peered":"2024-02-21T23:25:16.134602+0000","last_unstale":"2024-02-21T23:53:18.843551+0000","last_undegraded":"2024-02-21T23:53:18.843551+0000","last_fullsized":"2024-02-21T23:53:18.843551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'143","reported_seq":174,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.422758+0000","last_change":"2024-02-21T23:25:22.249606+0000","last_active":"2024-02-21T23:52:56.422758+0000","last_peered":"2024-02-21T23:52:56.422758+0000","last_clean":"2024-02-21T23:52:56.422758+0000","last_became_active":"2024-02-21T23:25:22.249214+0000","last_became_peered":"2024-02-21T23:25:22.249214+0000","last_unstale":"2024-02-21T23:52:56.422758+0000","last_undegraded":"2024-02-21T23:52:56.422758+0000","last_fullsized":"2024-02-21T23:52:56.422758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:36.929076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":6395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'33","reported_seq":70,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471655+0000","last_change":"2024-02-21T23:25:17.183290+0000","last_active":"2024-02-21T23:53:24.471655+0000","last_peered":"2024-02-21T23:53:24.471655+0000","last_clean":"2024-02-21T23:53:24.471655+0000","last_became_active":"2024-02-21T23:25:17.183047+0000","last_became_peered":"2024-02-21T23:25:17.183047+0000","last_unstale":"2024-02-21T23:53:24.471655+0000","last_undegraded":"2024-02-21T23:53:24.471655+0000","last_fullsized":"2024-02-21T23:53:24.471655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'1368","reported_seq":1407,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.000230+0000","last_change":"2024-02-21T23:25:16.146202+0000","last_active":"2024-02-21T23:53:23.000230+0000","last_peered":"2024-02-21T23:53:23.000230+0000","last_clean":"2024-02-21T23:53:23.000230+0000","last_became_active":"2024-02-21T23:25:16.145911+0000","last_became_peered":"2024-02-21T23:25:16.145911+0000","last_unstale":"2024-02-21T23:53:23.000230+0000","last_undegraded":"2024-02-21T23:53:23.000230+0000","last_fullsized":"2024-02-21T23:53:23.000230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3511,"num_write_kb":1242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'135","reported_seq":167,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700303+0000","last_change":"2024-02-21T23:25:22.243297+0000","last_active":"2024-02-21T23:52:56.700303+0000","last_peered":"2024-02-21T23:52:56.700303+0000","last_clean":"2024-02-21T23:52:56.700303+0000","last_became_active":"2024-02-21T23:25:22.242922+0000","last_became_peered":"2024-02-21T23:25:22.242922+0000","last_unstale":"2024-02-21T23:52:56.700303+0000","last_undegraded":"2024-02-21T23:52:56.700303+0000","last_fullsized":"2024-02-21T23:52:56.700303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:52:51.773574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":7447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"26'37","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.456470+0000","last_change":"2024-02-21T23:25:17.190577+0000","last_active":"2024-02-21T23:53:24.456470+0000","last_peered":"2024-02-21T23:53:24.456470+0000","last_clean":"2024-02-21T23:53:24.456470+0000","last_became_active":"2024-02-21T23:25:17.190332+0000","last_became_peered":"2024-02-21T23:25:17.190332+0000","last_unstale":"2024-02-21T23:53:24.456470+0000","last_undegraded":"2024-02-21T23:53:24.456470+0000","last_fullsized":"2024-02-21T23:53:24.456470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'3","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.853192+0000","last_change":"2024-02-21T23:25:16.134076+0000","last_active":"2024-02-21T23:53:15.853192+0000","last_peered":"2024-02-21T23:53:15.853192+0000","last_clean":"2024-02-21T23:53:15.853192+0000","last_became_active":"2024-02-21T23:25:16.133816+0000","last_became_peered":"2024-02-21T23:25:16.133816+0000","last_unstale":"2024-02-21T23:53:15.853192+0000","last_undegraded":"2024-02-21T23:53:15.853192+0000","last_fullsized":"2024-02-21T23:53:15.853192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'127","reported_seq":158,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.466825+0000","last_change":"2024-02-21T23:25:22.243360+0000","last_active":"2024-02-21T23:52:48.466825+0000","last_peered":"2024-02-21T23:52:48.466825+0000","last_clean":"2024-02-21T23:52:48.466825+0000","last_became_active":"2024-02-21T23:25:22.242958+0000","last_became_peered":"2024-02-21T23:25:22.242958+0000","last_unstale":"2024-02-21T23:52:48.466825+0000","last_undegraded":"2024-02-21T23:52:48.466825+0000","last_fullsized":"2024-02-21T23:52:48.466825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":7411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494810+0000","last_change":"2024-02-21T23:25:17.192402+0000","last_active":"2024-02-21T23:53:24.494810+0000","last_peered":"2024-02-21T23:53:24.494810+0000","last_clean":"2024-02-21T23:53:24.494810+0000","last_became_active":"2024-02-21T23:25:17.192138+0000","last_became_peered":"2024-02-21T23:25:17.192138+0000","last_unstale":"2024-02-21T23:53:24.494810+0000","last_undegraded":"2024-02-21T23:53:24.494810+0000","last_fullsized":"2024-02-21T23:53:24.494810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:03.326742+0000","last_change":"2024-02-21T23:25:16.145521+0000","last_active":"2024-02-21T23:53:03.326742+0000","last_peered":"2024-02-21T23:53:03.326742+0000","last_clean":"2024-02-21T23:53:03.326742+0000","last_became_active":"2024-02-21T23:25:16.145255+0000","last_became_peered":"2024-02-21T23:25:16.145255+0000","last_unstale":"2024-02-21T23:53:03.326742+0000","last_undegraded":"2024-02-21T23:53:03.326742+0000","last_fullsized":"2024-02-21T23:53:03.326742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'208","reported_seq":239,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.413727+0000","last_change":"2024-02-21T23:25:21.683849+0000","last_active":"2024-02-21T23:52:56.413727+0000","last_peered":"2024-02-21T23:52:56.413727+0000","last_clean":"2024-02-21T23:52:56.413727+0000","last_became_active":"2024-02-21T23:25:21.683379+0000","last_became_peered":"2024-02-21T23:25:21.683379+0000","last_unstale":"2024-02-21T23:52:56.413727+0000","last_undegraded":"2024-02-21T23:52:56.413727+0000","last_fullsized":"2024-02-21T23:52:56.413727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":191,"num_write_kb":16487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"26'31","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.482603+0000","last_change":"2024-02-21T23:25:17.166232+0000","last_active":"2024-02-21T23:53:24.482603+0000","last_peered":"2024-02-21T23:53:24.482603+0000","last_clean":"2024-02-21T23:53:24.482603+0000","last_became_active":"2024-02-21T23:25:17.165999+0000","last_became_peered":"2024-02-21T23:25:17.165999+0000","last_unstale":"2024-02-21T23:53:24.482603+0000","last_undegraded":"2024-02-21T23:53:24.482603+0000","last_fullsized":"2024-02-21T23:53:24.482603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:45.125986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:23.691688+0000","last_change":"2024-02-21T23:25:16.147082+0000","last_active":"2024-02-21T23:52:23.691688+0000","last_peered":"2024-02-21T23:52:23.691688+0000","last_clean":"2024-02-21T23:52:23.691688+0000","last_became_active":"2024-02-21T23:25:16.146797+0000","last_became_peered":"2024-02-21T23:25:16.146797+0000","last_unstale":"2024-02-21T23:52:23.691688+0000","last_undegraded":"2024-02-21T23:52:23.691688+0000","last_fullsized":"2024-02-21T23:52:23.691688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'125","reported_seq":150,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.407469+0000","last_change":"2024-02-21T23:25:57.583768+0000","last_active":"2024-02-21T23:52:56.407469+0000","last_peered":"2024-02-21T23:52:56.407469+0000","last_clean":"2024-02-21T23:52:56.407469+0000","last_became_active":"2024-02-21T23:25:57.583297+0000","last_became_peered":"2024-02-21T23:25:57.583297+0000","last_unstale":"2024-02-21T23:52:56.407469+0000","last_undegraded":"2024-02-21T23:52:56.407469+0000","last_fullsized":"2024-02-21T23:52:56.407469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:19:14.798657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":6113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,6],"acting":[0,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"26'33","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.454823+0000","last_change":"2024-02-21T23:25:17.190425+0000","last_active":"2024-02-21T23:53:24.454823+0000","last_peered":"2024-02-21T23:53:24.454823+0000","last_clean":"2024-02-21T23:53:24.454823+0000","last_became_active":"2024-02-21T23:25:17.190172+0000","last_became_peered":"2024-02-21T23:25:17.190172+0000","last_unstale":"2024-02-21T23:53:24.454823+0000","last_undegraded":"2024-02-21T23:53:24.454823+0000","last_fullsized":"2024-02-21T23:53:24.454823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'999","reported_seq":2032,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.855182+0000","last_change":"2024-02-21T23:25:16.108263+0000","last_active":"2024-02-21T23:53:23.855182+0000","last_peered":"2024-02-21T23:53:23.855182+0000","last_clean":"2024-02-21T23:53:23.855182+0000","last_became_active":"2024-02-21T23:25:16.107822+0000","last_became_peered":"2024-02-21T23:25:16.107822+0000","last_unstale":"2024-02-21T23:53:23.855182+0000","last_undegraded":"2024-02-21T23:53:23.855182+0000","last_fullsized":"2024-02-21T23:53:23.855182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":995,"num_write_kb":1061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'134","reported_seq":164,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.702471+0000","last_change":"2024-02-21T23:25:21.686034+0000","last_active":"2024-02-21T23:52:56.702471+0000","last_peered":"2024-02-21T23:52:56.702471+0000","last_clean":"2024-02-21T23:52:56.702471+0000","last_became_active":"2024-02-21T23:25:21.685546+0000","last_became_peered":"2024-02-21T23:25:21.685546+0000","last_unstale":"2024-02-21T23:52:56.702471+0000","last_undegraded":"2024-02-21T23:52:56.702471+0000","last_fullsized":"2024-02-21T23:52:56.702471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":10678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607087+0000","last_change":"2024-02-21T23:25:17.161738+0000","last_active":"2024-02-21T23:53:24.607087+0000","last_peered":"2024-02-21T23:53:24.607087+0000","last_clean":"2024-02-21T23:53:24.607087+0000","last_became_active":"2024-02-21T23:25:17.161459+0000","last_became_peered":"2024-02-21T23:25:17.161459+0000","last_unstale":"2024-02-21T23:53:24.607087+0000","last_undegraded":"2024-02-21T23:53:24.607087+0000","last_fullsized":"2024-02-21T23:53:24.607087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:31:58.434561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'541","reported_seq":581,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.846835+0000","last_change":"2024-02-21T23:25:16.147162+0000","last_active":"2024-02-21T23:53:18.846835+0000","last_peered":"2024-02-21T23:53:18.846835+0000","last_clean":"2024-02-21T23:53:18.846835+0000","last_became_active":"2024-02-21T23:25:16.146850+0000","last_became_peered":"2024-02-21T23:25:16.146850+0000","last_unstale":"2024-02-21T23:53:18.846835+0000","last_undegraded":"2024-02-21T23:53:18.846835+0000","last_fullsized":"2024-02-21T23:53:18.846835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'140","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.987450+0000","last_change":"2024-02-21T23:25:22.248109+0000","last_active":"2024-02-21T23:52:52.987450+0000","last_peered":"2024-02-21T23:52:52.987450+0000","last_clean":"2024-02-21T23:52:52.987450+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:52:52.987450+0000","last_undegraded":"2024-02-21T23:52:52.987450+0000","last_fullsized":"2024-02-21T23:52:52.987450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"26'35","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.466335+0000","last_change":"2024-02-21T23:25:57.600968+0000","last_active":"2024-02-21T23:53:24.466335+0000","last_peered":"2024-02-21T23:53:24.466335+0000","last_clean":"2024-02-21T23:53:24.466335+0000","last_became_active":"2024-02-21T23:25:57.600644+0000","last_became_peered":"2024-02-21T23:25:57.600644+0000","last_unstale":"2024-02-21T23:53:24.466335+0000","last_undegraded":"2024-02-21T23:53:24.466335+0000","last_fullsized":"2024-02-21T23:53:24.466335+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:13:09.640854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'5","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:10.587650+0000","last_change":"2024-02-21T23:25:16.127688+0000","last_active":"2024-02-21T23:53:10.587650+0000","last_peered":"2024-02-21T23:53:10.587650+0000","last_clean":"2024-02-21T23:53:10.587650+0000","last_became_active":"2024-02-21T23:25:16.127388+0000","last_became_peered":"2024-02-21T23:25:16.127388+0000","last_unstale":"2024-02-21T23:53:10.587650+0000","last_undegraded":"2024-02-21T23:53:10.587650+0000","last_fullsized":"2024-02-21T23:53:10.587650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:51:13.500334+0000","last_change":"2024-02-21T23:25:05.612817+0000","last_active":"2024-02-21T23:51:13.500334+0000","last_peered":"2024-02-21T23:51:13.500334+0000","last_clean":"2024-02-21T23:51:13.500334+0000","last_became_active":"2024-02-21T23:25:05.612483+0000","last_became_peered":"2024-02-21T23:25:05.612483+0000","last_unstale":"2024-02-21T23:51:13.500334+0000","last_undegraded":"2024-02-21T23:51:13.500334+0000","last_fullsized":"2024-02-21T23:51:13.500334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_clean_scrub_stamp":"2024-02-21T23:25:03.736732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:23:33.073040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'168","reported_seq":199,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.098108+0000","last_change":"2024-02-21T23:25:22.248174+0000","last_active":"2024-02-21T23:52:55.098108+0000","last_peered":"2024-02-21T23:52:55.098108+0000","last_clean":"2024-02-21T23:52:55.098108+0000","last_became_active":"2024-02-21T23:25:22.247834+0000","last_became_peered":"2024-02-21T23:25:22.247834+0000","last_unstale":"2024-02-21T23:52:55.098108+0000","last_undegraded":"2024-02-21T23:52:55.098108+0000","last_fullsized":"2024-02-21T23:52:55.098108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:13.953358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":10993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604646+0000","last_change":"2024-02-21T23:25:17.192575+0000","last_active":"2024-02-21T23:53:24.604646+0000","last_peered":"2024-02-21T23:53:24.604646+0000","last_clean":"2024-02-21T23:53:24.604646+0000","last_became_active":"2024-02-21T23:25:17.192293+0000","last_became_peered":"2024-02-21T23:25:17.192293+0000","last_unstale":"2024-02-21T23:53:24.604646+0000","last_undegraded":"2024-02-21T23:53:24.604646+0000","last_fullsized":"2024-02-21T23:53:24.604646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:57.046912+0000","last_change":"2024-02-21T23:25:16.128369+0000","last_active":"2024-02-21T23:52:57.046912+0000","last_peered":"2024-02-21T23:52:57.046912+0000","last_clean":"2024-02-21T23:52:57.046912+0000","last_became_active":"2024-02-21T23:25:16.128067+0000","last_became_peered":"2024-02-21T23:25:16.128067+0000","last_unstale":"2024-02-21T23:52:57.046912+0000","last_undegraded":"2024-02-21T23:52:57.046912+0000","last_fullsized":"2024-02-21T23:52:57.046912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'138","reported_seq":168,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.670157+0000","last_change":"2024-02-21T23:25:22.243282+0000","last_active":"2024-02-21T23:52:52.670157+0000","last_peered":"2024-02-21T23:52:52.670157+0000","last_clean":"2024-02-21T23:52:52.670157+0000","last_became_active":"2024-02-21T23:25:22.242916+0000","last_became_peered":"2024-02-21T23:25:22.242916+0000","last_unstale":"2024-02-21T23:52:52.670157+0000","last_undegraded":"2024-02-21T23:52:52.670157+0000","last_fullsized":"2024-02-21T23:52:52.670157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"26'40","reported_seq":76,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493486+0000","last_change":"2024-02-21T23:25:17.185418+0000","last_active":"2024-02-21T23:53:24.493486+0000","last_peered":"2024-02-21T23:53:24.493486+0000","last_clean":"2024-02-21T23:53:24.493486+0000","last_became_active":"2024-02-21T23:25:17.185119+0000","last_became_peered":"2024-02-21T23:25:17.185119+0000","last_unstale":"2024-02-21T23:53:24.493486+0000","last_undegraded":"2024-02-21T23:53:24.493486+0000","last_fullsized":"2024-02-21T23:53:24.493486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:18.471011+0000","last_change":"2024-02-21T23:25:16.151088+0000","last_active":"2024-02-21T23:52:18.471011+0000","last_peered":"2024-02-21T23:52:18.471011+0000","last_clean":"2024-02-21T23:52:18.471011+0000","last_became_active":"2024-02-21T23:25:16.150819+0000","last_became_peered":"2024-02-21T23:25:16.150819+0000","last_unstale":"2024-02-21T23:52:18.471011+0000","last_undegraded":"2024-02-21T23:52:18.471011+0000","last_fullsized":"2024-02-21T23:52:18.471011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'189","reported_seq":219,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.685417+0000","last_change":"2024-02-21T23:25:21.684592+0000","last_active":"2024-02-21T23:52:56.685417+0000","last_peered":"2024-02-21T23:52:56.685417+0000","last_clean":"2024-02-21T23:52:56.685417+0000","last_became_active":"2024-02-21T23:25:21.684192+0000","last_became_peered":"2024-02-21T23:25:21.684192+0000","last_unstale":"2024-02-21T23:52:56.685417+0000","last_undegraded":"2024-02-21T23:52:56.685417+0000","last_fullsized":"2024-02-21T23:52:56.685417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":12683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.606181+0000","last_change":"2024-02-21T23:25:17.184268+0000","last_active":"2024-02-21T23:53:24.606181+0000","last_peered":"2024-02-21T23:53:24.606181+0000","last_clean":"2024-02-21T23:53:24.606181+0000","last_became_active":"2024-02-21T23:25:17.184027+0000","last_became_peered":"2024-02-21T23:25:17.184027+0000","last_unstale":"2024-02-21T23:53:24.606181+0000","last_undegraded":"2024-02-21T23:53:24.606181+0000","last_fullsized":"2024-02-21T23:53:24.606181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'4","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:09.554814+0000","last_change":"2024-02-21T23:25:16.108099+0000","last_active":"2024-02-21T23:53:09.554814+0000","last_peered":"2024-02-21T23:53:09.554814+0000","last_clean":"2024-02-21T23:53:09.554814+0000","last_became_active":"2024-02-21T23:25:16.107815+0000","last_became_peered":"2024-02-21T23:25:16.107815+0000","last_unstale":"2024-02-21T23:53:09.554814+0000","last_undegraded":"2024-02-21T23:53:09.554814+0000","last_fullsized":"2024-02-21T23:53:09.554814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'131","reported_seq":161,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.464577+0000","last_change":"2024-02-21T23:25:22.248926+0000","last_active":"2024-02-21T23:52:48.464577+0000","last_peered":"2024-02-21T23:52:48.464577+0000","last_clean":"2024-02-21T23:52:48.464577+0000","last_became_active":"2024-02-21T23:25:22.248558+0000","last_became_peered":"2024-02-21T23:25:22.248558+0000","last_unstale":"2024-02-21T23:52:48.464577+0000","last_undegraded":"2024-02-21T23:52:48.464577+0000","last_fullsized":"2024-02-21T23:52:48.464577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"26'23","reported_seq":59,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605888+0000","last_change":"2024-02-21T23:25:17.183254+0000","last_active":"2024-02-21T23:53:24.605888+0000","last_peered":"2024-02-21T23:53:24.605888+0000","last_clean":"2024-02-21T23:53:24.605888+0000","last_became_active":"2024-02-21T23:25:17.183020+0000","last_became_peered":"2024-02-21T23:25:17.183020+0000","last_unstale":"2024-02-21T23:53:24.605888+0000","last_undegraded":"2024-02-21T23:53:24.605888+0000","last_fullsized":"2024-02-21T23:53:24.605888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:23.581289+0000","last_change":"2024-02-21T23:25:16.129131+0000","last_active":"2024-02-21T23:52:23.581289+0000","last_peered":"2024-02-21T23:52:23.581289+0000","last_clean":"2024-02-21T23:52:23.581289+0000","last_became_active":"2024-02-21T23:25:16.128853+0000","last_became_peered":"2024-02-21T23:25:16.128853+0000","last_unstale":"2024-02-21T23:52:23.581289+0000","last_undegraded":"2024-02-21T23:52:23.581289+0000","last_fullsized":"2024-02-21T23:52:23.581289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:29.302181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'163","reported_seq":193,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.422752+0000","last_change":"2024-02-21T23:25:21.684342+0000","last_active":"2024-02-21T23:52:56.422752+0000","last_peered":"2024-02-21T23:52:56.422752+0000","last_clean":"2024-02-21T23:52:56.422752+0000","last_became_active":"2024-02-21T23:25:21.683991+0000","last_became_peered":"2024-02-21T23:25:21.683991+0000","last_unstale":"2024-02-21T23:52:56.422752+0000","last_undegraded":"2024-02-21T23:52:56.422752+0000","last_fullsized":"2024-02-21T23:52:56.422752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:21:00.312314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":12715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"26'37","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.606217+0000","last_change":"2024-02-21T23:25:17.183442+0000","last_active":"2024-02-21T23:53:24.606217+0000","last_peered":"2024-02-21T23:53:24.606217+0000","last_clean":"2024-02-21T23:53:24.606217+0000","last_became_active":"2024-02-21T23:25:17.183057+0000","last_became_peered":"2024-02-21T23:25:17.183057+0000","last_unstale":"2024-02-21T23:53:24.606217+0000","last_undegraded":"2024-02-21T23:53:24.606217+0000","last_fullsized":"2024-02-21T23:53:24.606217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'13","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.623678+0000","last_change":"2024-02-21T23:25:16.145801+0000","last_active":"2024-02-21T23:53:18.623678+0000","last_peered":"2024-02-21T23:53:18.623678+0000","last_clean":"2024-02-21T23:53:18.623678+0000","last_became_active":"2024-02-21T23:25:16.145523+0000","last_became_peered":"2024-02-21T23:25:16.145523+0000","last_unstale":"2024-02-21T23:53:18.623678+0000","last_undegraded":"2024-02-21T23:53:18.623678+0000","last_fullsized":"2024-02-21T23:53:18.623678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'157","reported_seq":186,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.093033+0000","last_change":"2024-02-21T23:25:21.679191+0000","last_active":"2024-02-21T23:52:55.093033+0000","last_peered":"2024-02-21T23:52:55.093033+0000","last_clean":"2024-02-21T23:52:55.093033+0000","last_became_active":"2024-02-21T23:25:21.678715+0000","last_became_peered":"2024-02-21T23:25:21.678715+0000","last_unstale":"2024-02-21T23:52:55.093033+0000","last_undegraded":"2024-02-21T23:52:55.093033+0000","last_fullsized":"2024-02-21T23:52:55.093033+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":11459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"26'29","reported_seq":65,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.625866+0000","last_change":"2024-02-21T23:25:17.138401+0000","last_active":"2024-02-21T23:53:24.625866+0000","last_peered":"2024-02-21T23:53:24.625866+0000","last_clean":"2024-02-21T23:53:24.625866+0000","last_became_active":"2024-02-21T23:25:17.138136+0000","last_became_peered":"2024-02-21T23:25:17.138136+0000","last_unstale":"2024-02-21T23:53:24.625866+0000","last_undegraded":"2024-02-21T23:53:24.625866+0000","last_fullsized":"2024-02-21T23:53:24.625866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'8","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.890247+0000","last_change":"2024-02-21T23:25:16.130209+0000","last_active":"2024-02-21T23:53:15.890247+0000","last_peered":"2024-02-21T23:53:15.890247+0000","last_clean":"2024-02-21T23:53:15.890247+0000","last_became_active":"2024-02-21T23:25:16.129974+0000","last_became_peered":"2024-02-21T23:25:16.129974+0000","last_unstale":"2024-02-21T23:53:15.890247+0000","last_undegraded":"2024-02-21T23:53:15.890247+0000","last_fullsized":"2024-02-21T23:53:15.890247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'138","reported_seq":167,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700347+0000","last_change":"2024-02-21T23:25:22.243303+0000","last_active":"2024-02-21T23:52:56.700347+0000","last_peered":"2024-02-21T23:52:56.700347+0000","last_clean":"2024-02-21T23:52:56.700347+0000","last_became_active":"2024-02-21T23:25:22.242960+0000","last_became_peered":"2024-02-21T23:25:22.242960+0000","last_unstale":"2024-02-21T23:52:56.700347+0000","last_undegraded":"2024-02-21T23:52:56.700347+0000","last_fullsized":"2024-02-21T23:52:56.700347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:54:16.386354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":8319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"26'48","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605915+0000","last_change":"2024-02-21T23:25:17.184233+0000","last_active":"2024-02-21T23:53:24.605915+0000","last_peered":"2024-02-21T23:53:24.605915+0000","last_clean":"2024-02-21T23:53:24.605915+0000","last_became_active":"2024-02-21T23:25:17.183980+0000","last_became_peered":"2024-02-21T23:25:17.183980+0000","last_unstale":"2024-02-21T23:53:24.605915+0000","last_undegraded":"2024-02-21T23:53:24.605915+0000","last_fullsized":"2024-02-21T23:53:24.605915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'3","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:14.807121+0000","last_change":"2024-02-21T23:25:16.131147+0000","last_active":"2024-02-21T23:53:14.807121+0000","last_peered":"2024-02-21T23:53:14.807121+0000","last_clean":"2024-02-21T23:53:14.807121+0000","last_became_active":"2024-02-21T23:25:16.130553+0000","last_became_peered":"2024-02-21T23:25:16.130553+0000","last_unstale":"2024-02-21T23:53:14.807121+0000","last_undegraded":"2024-02-21T23:53:14.807121+0000","last_fullsized":"2024-02-21T23:53:14.807121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'97","reported_seq":127,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.566724+0000","last_change":"2024-02-21T23:25:22.243261+0000","last_active":"2024-02-21T23:52:56.566724+0000","last_peered":"2024-02-21T23:52:56.566724+0000","last_clean":"2024-02-21T23:52:56.566724+0000","last_became_active":"2024-02-21T23:25:22.242900+0000","last_became_peered":"2024-02-21T23:25:22.242900+0000","last_unstale":"2024-02-21T23:52:56.566724+0000","last_undegraded":"2024-02-21T23:52:56.566724+0000","last_fullsized":"2024-02-21T23:52:56.566724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:34:44.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":7038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'44","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607089+0000","last_change":"2024-02-21T23:25:17.183303+0000","last_active":"2024-02-21T23:53:24.607089+0000","last_peered":"2024-02-21T23:53:24.607089+0000","last_clean":"2024-02-21T23:53:24.607089+0000","last_became_active":"2024-02-21T23:25:17.183042+0000","last_became_peered":"2024-02-21T23:25:17.183042+0000","last_unstale":"2024-02-21T23:53:24.607089+0000","last_undegraded":"2024-02-21T23:53:24.607089+0000","last_fullsized":"2024-02-21T23:53:24.607089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'251","reported_seq":289,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.291094+0000","last_change":"2024-02-21T23:25:16.127609+0000","last_active":"2024-02-21T23:53:24.291094+0000","last_peered":"2024-02-21T23:53:24.291094+0000","last_clean":"2024-02-21T23:53:24.291094+0000","last_became_active":"2024-02-21T23:25:16.127375+0000","last_became_peered":"2024-02-21T23:25:16.127375+0000","last_unstale":"2024-02-21T23:53:24.291094+0000","last_undegraded":"2024-02-21T23:53:24.291094+0000","last_fullsized":"2024-02-21T23:53:24.291094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2561517,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":2649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'148","reported_seq":177,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.755700+0000","last_change":"2024-02-21T23:25:22.248329+0000","last_active":"2024-02-21T23:52:56.755700+0000","last_peered":"2024-02-21T23:52:56.755700+0000","last_clean":"2024-02-21T23:52:56.755700+0000","last_became_active":"2024-02-21T23:25:22.247889+0000","last_became_peered":"2024-02-21T23:25:22.247889+0000","last_unstale":"2024-02-21T23:52:56.755700+0000","last_undegraded":"2024-02-21T23:52:56.755700+0000","last_fullsized":"2024-02-21T23:52:56.755700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":13204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"26'37","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.527489+0000","last_change":"2024-02-21T23:25:57.527196+0000","last_active":"2024-02-21T23:53:24.527489+0000","last_peered":"2024-02-21T23:53:24.527489+0000","last_clean":"2024-02-21T23:53:24.527489+0000","last_became_active":"2024-02-21T23:25:57.526874+0000","last_became_peered":"2024-02-21T23:25:57.526874+0000","last_unstale":"2024-02-21T23:53:24.527489+0000","last_undegraded":"2024-02-21T23:53:24.527489+0000","last_fullsized":"2024-02-21T23:53:24.527489+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"26'835","reported_seq":873,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:16.019825+0000","last_change":"2024-02-21T23:25:16.116460+0000","last_active":"2024-02-21T23:53:16.019825+0000","last_peered":"2024-02-21T23:53:16.019825+0000","last_clean":"2024-02-21T23:53:16.019825+0000","last_became_active":"2024-02-21T23:25:16.116162+0000","last_became_peered":"2024-02-21T23:25:16.116162+0000","last_unstale":"2024-02-21T23:53:16.019825+0000","last_undegraded":"2024-02-21T23:53:16.019825+0000","last_fullsized":"2024-02-21T23:53:16.019825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'155","reported_seq":185,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.702447+0000","last_change":"2024-02-21T23:25:21.683199+0000","last_active":"2024-02-21T23:52:56.702447+0000","last_peered":"2024-02-21T23:52:56.702447+0000","last_clean":"2024-02-21T23:52:56.702447+0000","last_became_active":"2024-02-21T23:25:21.682425+0000","last_became_peered":"2024-02-21T23:25:21.682425+0000","last_unstale":"2024-02-21T23:52:56.702447+0000","last_undegraded":"2024-02-21T23:52:56.702447+0000","last_fullsized":"2024-02-21T23:52:56.702447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":9421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'32","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.487783+0000","last_change":"2024-02-21T23:25:17.184837+0000","last_active":"2024-02-21T23:53:24.487783+0000","last_peered":"2024-02-21T23:53:24.487783+0000","last_clean":"2024-02-21T23:53:24.487783+0000","last_became_active":"2024-02-21T23:25:17.184574+0000","last_became_peered":"2024-02-21T23:25:17.184574+0000","last_unstale":"2024-02-21T23:53:24.487783+0000","last_undegraded":"2024-02-21T23:53:24.487783+0000","last_fullsized":"2024-02-21T23:53:24.487783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'697","reported_seq":734,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.851348+0000","last_change":"2024-02-21T23:25:16.151031+0000","last_active":"2024-02-21T23:53:18.851348+0000","last_peered":"2024-02-21T23:53:18.851348+0000","last_clean":"2024-02-21T23:53:18.851348+0000","last_became_active":"2024-02-21T23:25:16.150767+0000","last_became_peered":"2024-02-21T23:25:16.150767+0000","last_unstale":"2024-02-21T23:53:18.851348+0000","last_undegraded":"2024-02-21T23:53:18.851348+0000","last_fullsized":"2024-02-21T23:53:18.851348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'152","reported_seq":182,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.422949+0000","last_change":"2024-02-21T23:25:21.683995+0000","last_active":"2024-02-21T23:52:56.422949+0000","last_peered":"2024-02-21T23:52:56.422949+0000","last_clean":"2024-02-21T23:52:56.422949+0000","last_became_active":"2024-02-21T23:25:21.683415+0000","last_became_peered":"2024-02-21T23:25:21.683415+0000","last_unstale":"2024-02-21T23:52:56.422949+0000","last_undegraded":"2024-02-21T23:52:56.422949+0000","last_fullsized":"2024-02-21T23:52:56.422949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"26'42","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604682+0000","last_change":"2024-02-21T23:25:17.187230+0000","last_active":"2024-02-21T23:53:24.604682+0000","last_peered":"2024-02-21T23:53:24.604682+0000","last_clean":"2024-02-21T23:53:24.604682+0000","last_became_active":"2024-02-21T23:25:17.186952+0000","last_became_peered":"2024-02-21T23:25:17.186952+0000","last_unstale":"2024-02-21T23:53:24.604682+0000","last_undegraded":"2024-02-21T23:53:24.604682+0000","last_fullsized":"2024-02-21T23:53:24.604682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:00.113126+0000","last_change":"2024-02-21T23:25:16.134722+0000","last_active":"2024-02-21T23:53:00.113126+0000","last_peered":"2024-02-21T23:53:00.113126+0000","last_clean":"2024-02-21T23:53:00.113126+0000","last_became_active":"2024-02-21T23:25:16.134458+0000","last_became_peered":"2024-02-21T23:25:16.134458+0000","last_unstale":"2024-02-21T23:53:00.113126+0000","last_undegraded":"2024-02-21T23:53:00.113126+0000","last_fullsized":"2024-02-21T23:53:00.113126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'139","reported_seq":169,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700599+0000","last_change":"2024-02-21T23:25:21.684109+0000","last_active":"2024-02-21T23:52:56.700599+0000","last_peered":"2024-02-21T23:52:56.700599+0000","last_clean":"2024-02-21T23:52:56.700599+0000","last_became_active":"2024-02-21T23:25:21.683705+0000","last_became_peered":"2024-02-21T23:25:21.683705+0000","last_unstale":"2024-02-21T23:52:56.700599+0000","last_undegraded":"2024-02-21T23:52:56.700599+0000","last_fullsized":"2024-02-21T23:52:56.700599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":10133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"26'42","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493527+0000","last_change":"2024-02-21T23:25:17.188374+0000","last_active":"2024-02-21T23:53:24.493527+0000","last_peered":"2024-02-21T23:53:24.493527+0000","last_clean":"2024-02-21T23:53:24.493527+0000","last_became_active":"2024-02-21T23:25:17.187792+0000","last_became_peered":"2024-02-21T23:25:17.187792+0000","last_unstale":"2024-02-21T23:53:24.493527+0000","last_undegraded":"2024-02-21T23:53:24.493527+0000","last_fullsized":"2024-02-21T23:53:24.493527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'5","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:01.175408+0000","last_change":"2024-02-21T23:25:16.129446+0000","last_active":"2024-02-21T23:53:01.175408+0000","last_peered":"2024-02-21T23:53:01.175408+0000","last_clean":"2024-02-21T23:53:01.175408+0000","last_became_active":"2024-02-21T23:25:16.129164+0000","last_became_peered":"2024-02-21T23:25:16.129164+0000","last_unstale":"2024-02-21T23:53:01.175408+0000","last_undegraded":"2024-02-21T23:53:01.175408+0000","last_fullsized":"2024-02-21T23:53:01.175408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'139","reported_seq":162,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.403580+0000","last_change":"2024-02-21T23:25:57.600307+0000","last_active":"2024-02-21T23:52:56.403580+0000","last_peered":"2024-02-21T23:52:56.403580+0000","last_clean":"2024-02-21T23:52:56.403580+0000","last_became_active":"2024-02-21T23:25:57.599867+0000","last_became_peered":"2024-02-21T23:25:57.599867+0000","last_unstale":"2024-02-21T23:52:56.403580+0000","last_undegraded":"2024-02-21T23:52:56.403580+0000","last_fullsized":"2024-02-21T23:52:56.403580+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":5393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,5],"acting":[2,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"26'153","reported_seq":185,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.092755+0000","last_change":"2024-02-21T23:25:22.248119+0000","last_active":"2024-02-21T23:52:55.092755+0000","last_peered":"2024-02-21T23:52:55.092755+0000","last_clean":"2024-02-21T23:52:55.092755+0000","last_became_active":"2024-02-21T23:25:22.247811+0000","last_became_peered":"2024-02-21T23:25:22.247811+0000","last_unstale":"2024-02-21T23:52:55.092755+0000","last_undegraded":"2024-02-21T23:52:55.092755+0000","last_fullsized":"2024-02-21T23:52:55.092755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":11393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'41","reported_seq":76,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.466267+0000","last_change":"2024-02-21T23:25:17.189176+0000","last_active":"2024-02-21T23:53:24.466267+0000","last_peered":"2024-02-21T23:53:24.466267+0000","last_clean":"2024-02-21T23:53:24.466267+0000","last_became_active":"2024-02-21T23:25:17.188885+0000","last_became_peered":"2024-02-21T23:25:17.188885+0000","last_unstale":"2024-02-21T23:53:24.466267+0000","last_undegraded":"2024-02-21T23:53:24.466267+0000","last_fullsized":"2024-02-21T23:53:24.466267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'4","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:04.260921+0000","last_change":"2024-02-21T23:25:16.132036+0000","last_active":"2024-02-21T23:53:04.260921+0000","last_peered":"2024-02-21T23:53:04.260921+0000","last_clean":"2024-02-21T23:53:04.260921+0000","last_became_active":"2024-02-21T23:25:16.131756+0000","last_became_peered":"2024-02-21T23:25:16.131756+0000","last_unstale":"2024-02-21T23:53:04.260921+0000","last_undegraded":"2024-02-21T23:53:04.260921+0000","last_fullsized":"2024-02-21T23:53:04.260921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"26'157","reported_seq":188,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.421511+0000","last_change":"2024-02-21T23:25:21.686133+0000","last_active":"2024-02-21T23:52:56.421511+0000","last_peered":"2024-02-21T23:52:56.421511+0000","last_clean":"2024-02-21T23:52:56.421511+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:52:56.421511+0000","last_undegraded":"2024-02-21T23:52:56.421511+0000","last_fullsized":"2024-02-21T23:52:56.421511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":15074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'44","reported_seq":79,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471213+0000","last_change":"2024-02-21T23:25:17.135158+0000","last_active":"2024-02-21T23:53:24.471213+0000","last_peered":"2024-02-21T23:53:24.471213+0000","last_clean":"2024-02-21T23:53:24.471213+0000","last_became_active":"2024-02-21T23:25:17.134888+0000","last_became_peered":"2024-02-21T23:25:17.134888+0000","last_unstale":"2024-02-21T23:53:24.471213+0000","last_undegraded":"2024-02-21T23:53:24.471213+0000","last_fullsized":"2024-02-21T23:53:24.471213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'5","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:10.598178+0000","last_change":"2024-02-21T23:25:16.116378+0000","last_active":"2024-02-21T23:53:10.598178+0000","last_peered":"2024-02-21T23:53:10.598178+0000","last_clean":"2024-02-21T23:53:10.598178+0000","last_became_active":"2024-02-21T23:25:16.116201+0000","last_became_peered":"2024-02-21T23:25:16.116201+0000","last_unstale":"2024-02-21T23:53:10.598178+0000","last_undegraded":"2024-02-21T23:53:10.598178+0000","last_fullsized":"2024-02-21T23:53:10.598178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'718","reported_seq":755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.456812+0000","last_change":"2024-02-21T23:25:16.132314+0000","last_active":"2024-02-21T23:52:48.456812+0000","last_peered":"2024-02-21T23:52:48.456812+0000","last_clean":"2024-02-21T23:52:48.456812+0000","last_became_active":"2024-02-21T23:25:16.132062+0000","last_became_peered":"2024-02-21T23:25:16.132062+0000","last_unstale":"2024-02-21T23:52:48.456812+0000","last_undegraded":"2024-02-21T23:52:48.456812+0000","last_fullsized":"2024-02-21T23:52:48.456812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":716,"num_write_kb":4453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'39","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607247+0000","last_change":"2024-02-21T23:25:17.161451+0000","last_active":"2024-02-21T23:53:24.607247+0000","last_peered":"2024-02-21T23:53:24.607247+0000","last_clean":"2024-02-21T23:53:24.607247+0000","last_became_active":"2024-02-21T23:25:17.161176+0000","last_became_peered":"2024-02-21T23:25:17.161176+0000","last_unstale":"2024-02-21T23:53:24.607247+0000","last_undegraded":"2024-02-21T23:53:24.607247+0000","last_fullsized":"2024-02-21T23:53:24.607247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'95","reported_seq":124,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:53.621659+0000","last_change":"2024-02-21T23:25:21.685724+0000","last_active":"2024-02-21T23:52:53.621659+0000","last_peered":"2024-02-21T23:52:53.621659+0000","last_clean":"2024-02-21T23:52:53.621659+0000","last_became_active":"2024-02-21T23:25:21.685375+0000","last_became_peered":"2024-02-21T23:25:21.685375+0000","last_unstale":"2024-02-21T23:52:53.621659+0000","last_undegraded":"2024-02-21T23:52:53.621659+0000","last_fullsized":"2024-02-21T23:52:53.621659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:56:28.795401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":7997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"26'688","reported_seq":725,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:06.343128+0000","last_change":"2024-02-21T23:25:16.134127+0000","last_active":"2024-02-21T23:53:06.343128+0000","last_peered":"2024-02-21T23:53:06.343128+0000","last_clean":"2024-02-21T23:53:06.343128+0000","last_became_active":"2024-02-21T23:25:16.133839+0000","last_became_peered":"2024-02-21T23:25:16.133839+0000","last_unstale":"2024-02-21T23:53:06.343128+0000","last_undegraded":"2024-02-21T23:53:06.343128+0000","last_fullsized":"2024-02-21T23:53:06.343128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'33","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.483697+0000","last_change":"2024-02-21T23:25:17.190455+0000","last_active":"2024-02-21T23:53:24.483697+0000","last_peered":"2024-02-21T23:53:24.483697+0000","last_clean":"2024-02-21T23:53:24.483697+0000","last_became_active":"2024-02-21T23:25:17.190194+0000","last_became_peered":"2024-02-21T23:25:17.190194+0000","last_unstale":"2024-02-21T23:53:24.483697+0000","last_undegraded":"2024-02-21T23:53:24.483697+0000","last_fullsized":"2024-02-21T23:53:24.483697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:03.451409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'159","reported_seq":195,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.988198+0000","last_change":"2024-02-21T23:25:57.526175+0000","last_active":"2024-02-21T23:52:52.988198+0000","last_peered":"2024-02-21T23:52:52.988198+0000","last_clean":"2024-02-21T23:52:52.988198+0000","last_became_active":"2024-02-21T23:25:57.525755+0000","last_became_peered":"2024-02-21T23:25:57.525755+0000","last_unstale":"2024-02-21T23:52:52.988198+0000","last_undegraded":"2024-02-21T23:52:52.988198+0000","last_fullsized":"2024-02-21T23:52:52.988198+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:18:17.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4,2],"acting":[3,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'48","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626470+0000","last_change":"2024-02-21T23:25:17.138039+0000","last_active":"2024-02-21T23:53:24.626470+0000","last_peered":"2024-02-21T23:53:24.626470+0000","last_clean":"2024-02-21T23:53:24.626470+0000","last_became_active":"2024-02-21T23:25:17.137717+0000","last_became_peered":"2024-02-21T23:25:17.137717+0000","last_unstale":"2024-02-21T23:53:24.626470+0000","last_undegraded":"2024-02-21T23:53:24.626470+0000","last_fullsized":"2024-02-21T23:53:24.626470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"26'48","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.529559+0000","last_change":"2024-02-21T23:25:17.194727+0000","last_active":"2024-02-21T23:53:24.529559+0000","last_peered":"2024-02-21T23:53:24.529559+0000","last_clean":"2024-02-21T23:53:24.529559+0000","last_became_active":"2024-02-21T23:25:17.194533+0000","last_became_peered":"2024-02-21T23:25:17.194533+0000","last_unstale":"2024-02-21T23:53:24.529559+0000","last_undegraded":"2024-02-21T23:53:24.529559+0000","last_fullsized":"2024-02-21T23:53:24.529559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"26'50","reported_seq":87,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493574+0000","last_change":"2024-02-21T23:25:17.191051+0000","last_active":"2024-02-21T23:53:24.493574+0000","last_peered":"2024-02-21T23:53:24.493574+0000","last_clean":"2024-02-21T23:53:24.493574+0000","last_became_active":"2024-02-21T23:25:17.190600+0000","last_became_peered":"2024-02-21T23:25:17.190600+0000","last_unstale":"2024-02-21T23:53:24.493574+0000","last_undegraded":"2024-02-21T23:53:24.493574+0000","last_fullsized":"2024-02-21T23:53:24.493574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.599153+0000","last_change":"2024-02-21T23:25:17.135311+0000","last_active":"2024-02-21T23:53:24.599153+0000","last_peered":"2024-02-21T23:53:24.599153+0000","last_clean":"2024-02-21T23:53:24.599153+0000","last_became_active":"2024-02-21T23:25:17.135011+0000","last_became_peered":"2024-02-21T23:25:17.135011+0000","last_unstale":"2024-02-21T23:53:24.599153+0000","last_undegraded":"2024-02-21T23:53:24.599153+0000","last_fullsized":"2024-02-21T23:53:24.599153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.472250+0000","last_change":"2024-02-21T23:25:17.161696+0000","last_active":"2024-02-21T23:53:24.472250+0000","last_peered":"2024-02-21T23:53:24.472250+0000","last_clean":"2024-02-21T23:53:24.472250+0000","last_became_active":"2024-02-21T23:25:17.161401+0000","last_became_peered":"2024-02-21T23:25:17.161401+0000","last_unstale":"2024-02-21T23:53:24.472250+0000","last_undegraded":"2024-02-21T23:53:24.472250+0000","last_fullsized":"2024-02-21T23:53:24.472250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"26'34","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.463178+0000","last_change":"2024-02-21T23:25:17.189731+0000","last_active":"2024-02-21T23:53:24.463178+0000","last_peered":"2024-02-21T23:53:24.463178+0000","last_clean":"2024-02-21T23:53:24.463178+0000","last_became_active":"2024-02-21T23:25:17.189484+0000","last_became_peered":"2024-02-21T23:25:17.189484+0000","last_unstale":"2024-02-21T23:53:24.463178+0000","last_undegraded":"2024-02-21T23:53:24.463178+0000","last_fullsized":"2024-02-21T23:53:24.463178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"26'45","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.628365+0000","last_change":"2024-02-21T23:25:17.184125+0000","last_active":"2024-02-21T23:53:24.628365+0000","last_peered":"2024-02-21T23:53:24.628365+0000","last_clean":"2024-02-21T23:53:24.628365+0000","last_became_active":"2024-02-21T23:25:17.183865+0000","last_became_peered":"2024-02-21T23:25:17.183865+0000","last_unstale":"2024-02-21T23:53:24.628365+0000","last_undegraded":"2024-02-21T23:53:24.628365+0000","last_fullsized":"2024-02-21T23:53:24.628365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'37","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.468066+0000","last_change":"2024-02-21T23:25:17.161482+0000","last_active":"2024-02-21T23:53:24.468066+0000","last_peered":"2024-02-21T23:53:24.468066+0000","last_clean":"2024-02-21T23:53:24.468066+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:24.468066+0000","last_undegraded":"2024-02-21T23:53:24.468066+0000","last_fullsized":"2024-02-21T23:53:24.468066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'32","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494217+0000","last_change":"2024-02-21T23:25:17.185691+0000","last_active":"2024-02-21T23:53:24.494217+0000","last_peered":"2024-02-21T23:53:24.494217+0000","last_clean":"2024-02-21T23:53:24.494217+0000","last_became_active":"2024-02-21T23:25:17.185420+0000","last_became_peered":"2024-02-21T23:25:17.185420+0000","last_unstale":"2024-02-21T23:53:24.494217+0000","last_undegraded":"2024-02-21T23:53:24.494217+0000","last_fullsized":"2024-02-21T23:53:24.494217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"26'46","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471138+0000","last_change":"2024-02-21T23:25:17.161321+0000","last_active":"2024-02-21T23:53:24.471138+0000","last_peered":"2024-02-21T23:53:24.471138+0000","last_clean":"2024-02-21T23:53:24.471138+0000","last_became_active":"2024-02-21T23:25:17.161010+0000","last_became_peered":"2024-02-21T23:25:17.161010+0000","last_unstale":"2024-02-21T23:53:24.471138+0000","last_undegraded":"2024-02-21T23:53:24.471138+0000","last_fullsized":"2024-02-21T23:53:24.471138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"26'39","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494124+0000","last_change":"2024-02-21T23:25:17.191072+0000","last_active":"2024-02-21T23:53:24.494124+0000","last_peered":"2024-02-21T23:53:24.494124+0000","last_clean":"2024-02-21T23:53:24.494124+0000","last_became_active":"2024-02-21T23:25:17.187741+0000","last_became_peered":"2024-02-21T23:25:17.187741+0000","last_unstale":"2024-02-21T23:53:24.494124+0000","last_undegraded":"2024-02-21T23:53:24.494124+0000","last_fullsized":"2024-02-21T23:53:24.494124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"26'53","reported_seq":89,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607148+0000","last_change":"2024-02-21T23:25:17.186616+0000","last_active":"2024-02-21T23:53:24.607148+0000","last_peered":"2024-02-21T23:53:24.607148+0000","last_clean":"2024-02-21T23:53:24.607148+0000","last_became_active":"2024-02-21T23:25:17.186108+0000","last_became_peered":"2024-02-21T23:25:17.186108+0000","last_unstale":"2024-02-21T23:53:24.607148+0000","last_undegraded":"2024-02-21T23:53:24.607148+0000","last_fullsized":"2024-02-21T23:53:24.607148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"26'43","reported_seq":79,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626297+0000","last_change":"2024-02-21T23:25:17.161428+0000","last_active":"2024-02-21T23:53:24.626297+0000","last_peered":"2024-02-21T23:53:24.626297+0000","last_clean":"2024-02-21T23:53:24.626297+0000","last_became_active":"2024-02-21T23:25:17.161140+0000","last_became_peered":"2024-02-21T23:25:17.161140+0000","last_unstale":"2024-02-21T23:53:24.626297+0000","last_undegraded":"2024-02-21T23:53:24.626297+0000","last_fullsized":"2024-02-21T23:53:24.626297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604720+0000","last_change":"2024-02-21T23:25:17.190666+0000","last_active":"2024-02-21T23:53:24.604720+0000","last_peered":"2024-02-21T23:53:24.604720+0000","last_clean":"2024-02-21T23:53:24.604720+0000","last_became_active":"2024-02-21T23:25:17.190420+0000","last_became_peered":"2024-02-21T23:25:17.190420+0000","last_unstale":"2024-02-21T23:53:24.604720+0000","last_undegraded":"2024-02-21T23:53:24.604720+0000","last_fullsized":"2024-02-21T23:53:24.604720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"26'48","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.531320+0000","last_change":"2024-02-21T23:25:17.189583+0000","last_active":"2024-02-21T23:53:24.531320+0000","last_peered":"2024-02-21T23:53:24.531320+0000","last_clean":"2024-02-21T23:53:24.531320+0000","last_became_active":"2024-02-21T23:25:17.189302+0000","last_became_peered":"2024-02-21T23:25:17.189302+0000","last_unstale":"2024-02-21T23:53:24.531320+0000","last_undegraded":"2024-02-21T23:53:24.531320+0000","last_fullsized":"2024-02-21T23:53:24.531320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"26'41","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493492+0000","last_change":"2024-02-21T23:25:17.188113+0000","last_active":"2024-02-21T23:53:24.493492+0000","last_peered":"2024-02-21T23:53:24.493492+0000","last_clean":"2024-02-21T23:53:24.493492+0000","last_became_active":"2024-02-21T23:25:17.187811+0000","last_became_peered":"2024-02-21T23:25:17.187811+0000","last_unstale":"2024-02-21T23:53:24.493492+0000","last_undegraded":"2024-02-21T23:53:24.493492+0000","last_fullsized":"2024-02-21T23:53:24.493492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"26'44","reported_seq":65,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.530148+0000","last_change":"2024-02-21T23:25:57.525755+0000","last_active":"2024-02-21T23:53:24.530148+0000","last_peered":"2024-02-21T23:53:24.530148+0000","last_clean":"2024-02-21T23:53:24.530148+0000","last_became_active":"2024-02-21T23:25:57.525443+0000","last_became_peered":"2024-02-21T23:25:57.525443+0000","last_unstale":"2024-02-21T23:53:24.530148+0000","last_undegraded":"2024-02-21T23:53:24.530148+0000","last_fullsized":"2024-02-21T23:53:24.530148+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.457278+0000","last_change":"2024-02-21T23:25:17.186378+0000","last_active":"2024-02-21T23:53:24.457278+0000","last_peered":"2024-02-21T23:53:24.457278+0000","last_clean":"2024-02-21T23:53:24.457278+0000","last_became_active":"2024-02-21T23:25:17.186086+0000","last_became_peered":"2024-02-21T23:25:17.186086+0000","last_unstale":"2024-02-21T23:53:24.457278+0000","last_undegraded":"2024-02-21T23:53:24.457278+0000","last_fullsized":"2024-02-21T23:53:24.457278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"26'41","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.602175+0000","last_change":"2024-02-21T23:25:17.190934+0000","last_active":"2024-02-21T23:53:24.602175+0000","last_peered":"2024-02-21T23:53:24.602175+0000","last_clean":"2024-02-21T23:53:24.602175+0000","last_became_active":"2024-02-21T23:25:17.190680+0000","last_became_peered":"2024-02-21T23:25:17.190680+0000","last_unstale":"2024-02-21T23:53:24.602175+0000","last_undegraded":"2024-02-21T23:53:24.602175+0000","last_fullsized":"2024-02-21T23:53:24.602175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:22:04.238374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"26'38","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.473047+0000","last_change":"2024-02-21T23:25:17.135140+0000","last_active":"2024-02-21T23:53:24.473047+0000","last_peered":"2024-02-21T23:53:24.473047+0000","last_clean":"2024-02-21T23:53:24.473047+0000","last_became_active":"2024-02-21T23:25:17.134866+0000","last_became_peered":"2024-02-21T23:25:17.134866+0000","last_unstale":"2024-02-21T23:53:24.473047+0000","last_undegraded":"2024-02-21T23:53:24.473047+0000","last_fullsized":"2024-02-21T23:53:24.473047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'27","reported_seq":62,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.488701+0000","last_change":"2024-02-21T23:25:17.162204+0000","last_active":"2024-02-21T23:53:24.488701+0000","last_peered":"2024-02-21T23:53:24.488701+0000","last_clean":"2024-02-21T23:53:24.488701+0000","last_became_active":"2024-02-21T23:25:17.161807+0000","last_became_peered":"2024-02-21T23:25:17.161807+0000","last_unstale":"2024-02-21T23:53:24.488701+0000","last_undegraded":"2024-02-21T23:53:24.488701+0000","last_fullsized":"2024-02-21T23:53:24.488701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"26'36","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626279+0000","last_change":"2024-02-21T23:25:17.188932+0000","last_active":"2024-02-21T23:53:24.626279+0000","last_peered":"2024-02-21T23:53:24.626279+0000","last_clean":"2024-02-21T23:53:24.626279+0000","last_became_active":"2024-02-21T23:25:17.188646+0000","last_became_peered":"2024-02-21T23:25:17.188646+0000","last_unstale":"2024-02-21T23:53:24.626279+0000","last_undegraded":"2024-02-21T23:53:24.626279+0000","last_fullsized":"2024-02-21T23:53:24.626279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"26'51","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.628333+0000","last_change":"2024-02-21T23:25:17.188333+0000","last_active":"2024-02-21T23:53:24.628333+0000","last_peered":"2024-02-21T23:53:24.628333+0000","last_clean":"2024-02-21T23:53:24.628333+0000","last_became_active":"2024-02-21T23:25:17.188066+0000","last_became_peered":"2024-02-21T23:25:17.188066+0000","last_unstale":"2024-02-21T23:53:24.628333+0000","last_undegraded":"2024-02-21T23:53:24.628333+0000","last_fullsized":"2024-02-21T23:53:24.628333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"26'39","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605416+0000","last_change":"2024-02-21T23:25:17.187549+0000","last_active":"2024-02-21T23:53:24.605416+0000","last_peered":"2024-02-21T23:53:24.605416+0000","last_clean":"2024-02-21T23:53:24.605416+0000","last_became_active":"2024-02-21T23:25:17.187297+0000","last_became_peered":"2024-02-21T23:25:17.187297+0000","last_unstale":"2024-02-21T23:53:24.605416+0000","last_undegraded":"2024-02-21T23:53:24.605416+0000","last_fullsized":"2024-02-21T23:53:24.605416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"26'30","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.535065+0000","last_change":"2024-02-21T23:25:57.526373+0000","last_active":"2024-02-21T23:53:24.535065+0000","last_peered":"2024-02-21T23:53:24.535065+0000","last_clean":"2024-02-21T23:53:24.535065+0000","last_became_active":"2024-02-21T23:25:57.526068+0000","last_became_peered":"2024-02-21T23:25:57.526068+0000","last_unstale":"2024-02-21T23:53:24.535065+0000","last_undegraded":"2024-02-21T23:53:24.535065+0000","last_fullsized":"2024-02-21T23:53:24.535065+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"26'37","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.597841+0000","last_change":"2024-02-21T23:25:17.183264+0000","last_active":"2024-02-21T23:53:24.597841+0000","last_peered":"2024-02-21T23:53:24.597841+0000","last_clean":"2024-02-21T23:53:24.597841+0000","last_became_active":"2024-02-21T23:25:17.182924+0000","last_became_peered":"2024-02-21T23:25:17.182924+0000","last_unstale":"2024-02-21T23:53:24.597841+0000","last_undegraded":"2024-02-21T23:53:24.597841+0000","last_fullsized":"2024-02-21T23:53:24.597841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"26'38","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494000+0000","last_change":"2024-02-21T23:25:17.190841+0000","last_active":"2024-02-21T23:53:24.494000+0000","last_peered":"2024-02-21T23:53:24.494000+0000","last_clean":"2024-02-21T23:53:24.494000+0000","last_became_active":"2024-02-21T23:25:17.190276+0000","last_became_peered":"2024-02-21T23:25:17.190276+0000","last_unstale":"2024-02-21T23:53:24.494000+0000","last_undegraded":"2024-02-21T23:53:24.494000+0000","last_fullsized":"2024-02-21T23:53:24.494000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'36","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.601860+0000","last_change":"2024-02-21T23:25:17.189872+0000","last_active":"2024-02-21T23:53:24.601860+0000","last_peered":"2024-02-21T23:53:24.601860+0000","last_clean":"2024-02-21T23:53:24.601860+0000","last_became_active":"2024-02-21T23:25:17.189614+0000","last_became_peered":"2024-02-21T23:25:17.189614+0000","last_unstale":"2024-02-21T23:53:24.601860+0000","last_undegraded":"2024-02-21T23:53:24.601860+0000","last_fullsized":"2024-02-21T23:53:24.601860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"26'25","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493930+0000","last_change":"2024-02-21T23:25:57.600458+0000","last_active":"2024-02-21T23:53:24.493930+0000","last_peered":"2024-02-21T23:53:24.493930+0000","last_clean":"2024-02-21T23:53:24.493930+0000","last_became_active":"2024-02-21T23:25:57.600094+0000","last_became_peered":"2024-02-21T23:25:57.600094+0000","last_unstale":"2024-02-21T23:53:24.493930+0000","last_undegraded":"2024-02-21T23:53:24.493930+0000","last_fullsized":"2024-02-21T23:53:24.493930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:20:16.177299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"26'44","reported_seq":81,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.602153+0000","last_change":"2024-02-21T23:25:17.191159+0000","last_active":"2024-02-21T23:53:24.602153+0000","last_peered":"2024-02-21T23:53:24.602153+0000","last_clean":"2024-02-21T23:53:24.602153+0000","last_became_active":"2024-02-21T23:25:17.190576+0000","last_became_peered":"2024-02-21T23:25:17.190576+0000","last_unstale":"2024-02-21T23:53:24.602153+0000","last_undegraded":"2024-02-21T23:53:24.602153+0000","last_fullsized":"2024-02-21T23:53:24.602153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"26'34","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.452090+0000","last_change":"2024-02-21T23:25:17.189287+0000","last_active":"2024-02-21T23:53:24.452090+0000","last_peered":"2024-02-21T23:53:24.452090+0000","last_clean":"2024-02-21T23:53:24.452090+0000","last_became_active":"2024-02-21T23:25:17.189029+0000","last_became_peered":"2024-02-21T23:25:17.189029+0000","last_unstale":"2024-02-21T23:53:24.452090+0000","last_undegraded":"2024-02-21T23:53:24.452090+0000","last_fullsized":"2024-02-21T23:53:24.452090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605166+0000","last_change":"2024-02-21T23:25:17.183737+0000","last_active":"2024-02-21T23:53:24.605166+0000","last_peered":"2024-02-21T23:53:24.605166+0000","last_clean":"2024-02-21T23:53:24.605166+0000","last_became_active":"2024-02-21T23:25:17.183410+0000","last_became_peered":"2024-02-21T23:25:17.183410+0000","last_unstale":"2024-02-21T23:53:24.605166+0000","last_undegraded":"2024-02-21T23:53:24.605166+0000","last_fullsized":"2024-02-21T23:53:24.605166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4087,"num_write_kb":323834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4616,"ondisk_log_size":4616,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2419,"ondisk_log_size":2419,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":40532172,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":11557,"num_write_kb":45747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55631872,"data_stored":81064344,"data_compressed":7543912,"data_compressed_allocated":11722752,"data_compressed_original":37216256,"omap_allocated":0,"internal_metadata":0},"log_size":9530,"ondisk_log_size":9530,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575193,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78552,"kb_used_data":16848,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94293288,"statfs":{"total":96636764160,"available":96556326912,"internally_reserved":0,"allocated":17252352,"data_stored":25871404,"data_compressed":2449552,"data_compressed_allocated":3768320,"data_compressed_original":12558336,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575193,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61012,"kb_used_data":11788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310828,"statfs":{"total":96636764160,"available":96574287872,"internally_reserved":0,"allocated":12070912,"data_stored":16892518,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575193,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":114720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575193,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60916,"kb_used_data":7532,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310924,"statfs":{"total":96636764160,"available":96574386176,"internally_reserved":0,"allocated":7712768,"data_stored":11064847,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575193,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58952,"kb_used_data":9728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312888,"statfs":{"total":96636764160,"available":96576397312,"internally_reserved":0,"allocated":9961472,"data_stored":12918697,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575193,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":495616,"data_stored":335113,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575193,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47824,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324016,"statfs":{"total":96636764160,"available":96587792384,"internally_reserved":0,"allocated":2826240,"data_stored":4309094,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575193,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60600,"kb_used_data":7216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311240,"statfs":{"total":96636764160,"available":96574709760,"internally_reserved":0,"allocated":7389184,"data_stored":11655727,"data_compressed":1158786,"data_compressed_allocated":1720320,"data_compressed_original":6168576,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7045120,"data_stored":10950729,"data_compressed":1126387,"data_compressed_allocated":1675264,"data_compressed_original":5595136,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2547712,"data_stored":4194464,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":220483,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9682944,"data_stored":12804067,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7434240,"data_stored":10950217,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11792384,"data_stored":16777888,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16908288,"data_stored":25166406,"data_compressed":2417153,"data_compressed_allocated":3723264,"data_compressed_original":11984896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-21T23:53:31.827 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-02-21T23:53:31.828 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-21T23:53:31.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.969+0000 7f16db91e640 1 Processor -- start 2024-02-21T23:53:31.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.969+0000 7f16db91e640 1 -- start start 2024-02-21T23:53:31.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.970+0000 7f16db91e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f16d415bec0 0x7f16d4071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.970+0000 7f16db91e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d4158230 con 0x7f16d40c53a0 2024-02-21T23:53:31.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.970+0000 7f16db91e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d4158510 con 0x7f16d40c4fc0 2024-02-21T23:53:31.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.970+0000 7f16db91e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d4158eb0 con 0x7f16d415bec0 2024-02-21T23:53:31.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.971+0000 7f16d96ba640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f16d40c4fc0 0x7f16d406f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57446/0 (socket says 172.21.15.133:57446) 2024-02-21T23:53:31.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.971+0000 7f16d96ba640 1 -- 172.21.15.133:0/2185625642 learned_addr learned my addr 172.21.15.133:0/2185625642 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:31.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.971+0000 7f16d9ebb640 1 --2- 172.21.15.133:0/2185625642 >> v2:172.21.15.133:3300/0 conn(0x7f16d415bec0 0x7f16d4071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.973+0000 7f16cbfff640 1 -- 172.21.15.133:0/2185625642 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317828513 0 0) 0x7f16d4158510 con 0x7f16d40c4fc0 2024-02-21T23:53:31.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.973+0000 7f16cbfff640 1 -- 172.21.15.133:0/2185625642 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c0003520 con 0x7f16d40c4fc0 2024-02-21T23:53:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.974+0000 7f16d9ebb640 1 -- 172.21.15.133:0/2185625642 >> v1:172.21.15.142:6789/0 conn(0x7f16d40c4fc0 legacy=0x7f16d406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.974+0000 7f16d9ebb640 1 -- 172.21.15.133:0/2185625642 >> v1:172.21.15.133:6789/0 conn(0x7f16d40c53a0 legacy=0x7f16d415b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.974+0000 7f16d9ebb640 1 -- 172.21.15.133:0/2185625642 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d41587f0 con 0x7f16d415bec0 2024-02-21T23:53:31.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.979+0000 7f16d9ebb640 1 --2- 172.21.15.133:0/2185625642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d415bec0 0x7f16d4071b60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0097a0 tx=0x7f16cc02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9ec38199f222b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:31.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.982+0000 7f16cbfff640 1 -- 172.21.15.133:0/2185625642 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc02f810 con 0x7f16d415bec0 2024-02-21T23:53:31.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.983+0000 7f16cbfff640 1 -- 172.21.15.133:0/2185625642 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc02f9b0 con 0x7f16d415bec0 2024-02-21T23:53:31.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.983+0000 7f16cbfff640 1 -- 172.21.15.133:0/2185625642 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc02fd80 con 0x7f16d415bec0 2024-02-21T23:53:31.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.983+0000 7f16db91e640 1 -- 172.21.15.133:0/2185625642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d415bec0 msgr2=0x7f16d4071b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.983+0000 7f16db91e640 1 --2- 172.21.15.133:0/2185625642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d415bec0 0x7f16d4071b60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0097a0 tx=0x7f16cc02b9f0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.983+0000 7f16db91e640 1 -- 172.21.15.133:0/2185625642 shutdown_connections 2024-02-21T23:53:31.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.983+0000 7f16db91e640 1 --2- 172.21.15.133:0/2185625642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d415bec0 0x7f16d4071b60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.983+0000 7f16db91e640 1 -- 172.21.15.133:0/2185625642 >> 172.21.15.133:0/2185625642 conn(0x7f16d406aec0 msgr2=0x7f16d406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:31.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.983+0000 7f16db91e640 1 -- 172.21.15.133:0/2185625642 shutdown_connections 2024-02-21T23:53:31.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.983+0000 7f16db91e640 1 -- 172.21.15.133:0/2185625642 wait complete. 2024-02-21T23:53:31.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.985+0000 7f16db91e640 1 Processor -- start 2024-02-21T23:53:31.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.985+0000 7f16db91e640 1 -- start start 2024-02-21T23:53:31.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.985+0000 7f16db91e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d40c4fc0 0x7f16d414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.985+0000 7f16db91e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d40c53a0 0x7f16d414c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.985+0000 7f16db91e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16d415bec0 0x7f16d414cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:31.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.985+0000 7f16db91e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d406f8d0 con 0x7f16d40c53a0 2024-02-21T23:53:31.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.985+0000 7f16db91e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d4157f20 con 0x7f16d40c4fc0 2024-02-21T23:53:31.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.985+0000 7f16db91e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d406fa50 con 0x7f16d415bec0 2024-02-21T23:53:31.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.986+0000 7f16d8eb9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d40c53a0 0x7f16d414c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.986+0000 7f16d9ebb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16d415bec0 0x7f16d414cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.986+0000 7f16d8eb9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d40c53a0 0x7f16d414c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36118/0 (socket says 172.21.15.133:36118) 2024-02-21T23:53:31.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.986+0000 7f16d8eb9640 1 -- 172.21.15.133:0/1857751870 learned_addr learned my addr 172.21.15.133:0/1857751870 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:31.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.987+0000 7f16d96ba640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d40c4fc0 0x7f16d414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:31.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.989+0000 7f16d8eb9640 1 -- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16d415bec0 msgr2=0x7f16d414cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.989+0000 7f16d8eb9640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16d415bec0 0x7f16d414cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.989+0000 7f16d8eb9640 1 -- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d40c4fc0 msgr2=0x7f16d414f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:31.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.989+0000 7f16d8eb9640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d40c4fc0 0x7f16d414f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:31.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.989+0000 7f16d8eb9640 1 -- 172.21.15.133:0/1857751870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4148470 con 0x7f16d40c53a0 2024-02-21T23:53:31.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.992+0000 7f16d8eb9640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d40c53a0 0x7f16d414c5a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f16c400c3a0 tx=0x7f16c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:31.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.993+0000 7f16c9ffb640 1 -- 172.21.15.133:0/1857751870 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c40052d0 con 0x7f16d40c53a0 2024-02-21T23:53:31.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.994+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d4148750 con 0x7f16d40c53a0 2024-02-21T23:53:31.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.994+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d4148d10 con 0x7f16d40c53a0 2024-02-21T23:53:31.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:31.995+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d406a130 con 0x7f16d40c53a0 2024-02-21T23:53:32.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.240+0000 7f16c9ffb640 1 -- 172.21.15.133:0/1857751870 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c4005470 con 0x7f16d40c53a0 2024-02-21T23:53:32.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.240+0000 7f16c9ffb640 1 -- 172.21.15.133:0/1857751870 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4020060 con 0x7f16d40c53a0 2024-02-21T23:53:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.241+0000 7f16c9ffb640 1 -- 172.21.15.133:0/1857751870 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16c4007260 con 0x7f16d40c53a0 2024-02-21T23:53:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.241+0000 7f16c9ffb640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f16b806b4d0 0x7f16b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.241+0000 7f16d96ba640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f16b806b4d0 0x7f16b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.242+0000 7f16d96ba640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f16b806b4d0 0x7f16b806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16d0001560 tx=0x7f16d0001a90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:32.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.256+0000 7f16c9ffb640 1 -- 172.21.15.133:0/1857751870 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f16c40562f0 con 0x7f16d40c53a0 2024-02-21T23:53:32.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.256+0000 7f16c9ffb640 1 -- 172.21.15.133:0/1857751870 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f16c4058250 con 0x7f16d40c53a0 2024-02-21T23:53:32.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.426+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16d4157d00 con 0x7f16b806b4d0 2024-02-21T23:53:32.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.434+0000 7f16c9ffb640 1 -- 172.21.15.133:0/1857751870 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+333996 (secure 0 0 0) 0x7f16d4157d00 con 0x7f16b806b4d0 2024-02-21T23:53:32.433 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:53:32.435 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-02-21T23:53:32.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.437+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f16b806b4d0 msgr2=0x7f16b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f16b806b4d0 0x7f16b806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16d0001560 tx=0x7f16d0001a90 comp rx=0 tx=0).stop 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d40c53a0 msgr2=0x7f16d414c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d40c53a0 0x7f16d414c5a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f16c400c3a0 tx=0x7f16c400c860 comp rx=0 tx=0).stop 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 shutdown_connections 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f16b806b4d0 0x7f16b806d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16d415bec0 0x7f16d414cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d40c53a0 0x7f16d414c5a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 --2- 172.21.15.133:0/1857751870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d40c4fc0 0x7f16d414f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 >> 172.21.15.133:0/1857751870 conn(0x7f16d406aec0 msgr2=0x7f16d4151fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 shutdown_connections 2024-02-21T23:53:32.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.438+0000 7f16db91e640 1 -- 172.21.15.133:0/1857751870 wait complete. 2024-02-21T23:53:32.444 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":873,"stamp":"2024-02-21T23:53:31.042599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":41122540,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":1110,"num_read_kb":1160,"num_write":15803,"num_write_kb":371997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16682,"ondisk_log_size":16682,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":426808,"kb_used_data":56632,"kb_used_omap":12,"kb_used_meta":370099,"kb_avail":754547912,"statfs":{"total":773094113280,"available":772657061888,"internally_reserved":0,"allocated":57991168,"data_stored":83162120,"data_compressed":7608710,"data_compressed_allocated":11812864,"data_compressed_original":38363136,"omap_allocated":12750,"internal_metadata":378981938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14811,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":15,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002580"},"pg_stats":[{"pgid":"2.1f","version":"26'604","reported_seq":643,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:17.860114+0000","last_change":"2024-02-21T23:25:16.134390+0000","last_active":"2024-02-21T23:53:17.860114+0000","last_peered":"2024-02-21T23:53:17.860114+0000","last_clean":"2024-02-21T23:53:17.860114+0000","last_became_active":"2024-02-21T23:25:16.134112+0000","last_became_peered":"2024-02-21T23:25:16.134112+0000","last_unstale":"2024-02-21T23:53:17.860114+0000","last_undegraded":"2024-02-21T23:53:17.860114+0000","last_fullsized":"2024-02-21T23:53:17.860114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'30","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.462957+0000","last_change":"2024-02-21T23:25:17.135738+0000","last_active":"2024-02-21T23:53:24.462957+0000","last_peered":"2024-02-21T23:53:24.462957+0000","last_clean":"2024-02-21T23:53:24.462957+0000","last_became_active":"2024-02-21T23:25:17.135586+0000","last_became_peered":"2024-02-21T23:25:17.135586+0000","last_unstale":"2024-02-21T23:53:24.462957+0000","last_undegraded":"2024-02-21T23:53:24.462957+0000","last_fullsized":"2024-02-21T23:53:24.462957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'161","reported_seq":190,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.384215+0000","last_change":"2024-02-21T23:25:22.248962+0000","last_active":"2024-02-21T23:52:56.384215+0000","last_peered":"2024-02-21T23:52:56.384215+0000","last_clean":"2024-02-21T23:52:56.384215+0000","last_became_active":"2024-02-21T23:25:22.248610+0000","last_became_peered":"2024-02-21T23:25:22.248610+0000","last_unstale":"2024-02-21T23:52:56.384215+0000","last_undegraded":"2024-02-21T23:52:56.384215+0000","last_fullsized":"2024-02-21T23:52:56.384215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:45.939036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":10997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'750","reported_seq":790,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:05.342762+0000","last_change":"2024-02-21T23:25:16.123635+0000","last_active":"2024-02-21T23:53:05.342762+0000","last_peered":"2024-02-21T23:53:05.342762+0000","last_clean":"2024-02-21T23:53:05.342762+0000","last_became_active":"2024-02-21T23:25:16.123374+0000","last_became_peered":"2024-02-21T23:25:16.123374+0000","last_unstale":"2024-02-21T23:53:05.342762+0000","last_undegraded":"2024-02-21T23:53:05.342762+0000","last_fullsized":"2024-02-21T23:53:05.342762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":4534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604770+0000","last_change":"2024-02-21T23:25:17.192858+0000","last_active":"2024-02-21T23:53:24.604770+0000","last_peered":"2024-02-21T23:53:24.604770+0000","last_clean":"2024-02-21T23:53:24.604770+0000","last_became_active":"2024-02-21T23:25:17.192626+0000","last_became_peered":"2024-02-21T23:25:17.192626+0000","last_unstale":"2024-02-21T23:53:24.604770+0000","last_undegraded":"2024-02-21T23:53:24.604770+0000","last_fullsized":"2024-02-21T23:53:24.604770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'132","reported_seq":168,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.423235+0000","last_change":"2024-02-21T23:25:57.583837+0000","last_active":"2024-02-21T23:52:56.423235+0000","last_peered":"2024-02-21T23:52:56.423235+0000","last_clean":"2024-02-21T23:52:56.423235+0000","last_became_active":"2024-02-21T23:25:57.583416+0000","last_became_peered":"2024-02-21T23:25:57.583416+0000","last_unstale":"2024-02-21T23:52:56.423235+0000","last_undegraded":"2024-02-21T23:52:56.423235+0000","last_fullsized":"2024-02-21T23:52:56.423235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:25:24.780185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":6267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"26'1343","reported_seq":1380,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.892783+0000","last_change":"2024-02-21T23:25:16.134805+0000","last_active":"2024-02-21T23:53:15.892783+0000","last_peered":"2024-02-21T23:53:15.892783+0000","last_clean":"2024-02-21T23:53:15.892783+0000","last_became_active":"2024-02-21T23:25:16.134533+0000","last_became_peered":"2024-02-21T23:25:16.134533+0000","last_unstale":"2024-02-21T23:53:15.892783+0000","last_undegraded":"2024-02-21T23:53:15.892783+0000","last_fullsized":"2024-02-21T23:53:15.892783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1334,"num_write_kb":8974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'32","reported_seq":67,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494748+0000","last_change":"2024-02-21T23:25:17.182131+0000","last_active":"2024-02-21T23:53:24.494748+0000","last_peered":"2024-02-21T23:53:24.494748+0000","last_clean":"2024-02-21T23:53:24.494748+0000","last_became_active":"2024-02-21T23:25:17.181886+0000","last_became_peered":"2024-02-21T23:25:17.181886+0000","last_unstale":"2024-02-21T23:53:24.494748+0000","last_undegraded":"2024-02-21T23:53:24.494748+0000","last_fullsized":"2024-02-21T23:53:24.494748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'141","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.423026+0000","last_change":"2024-02-21T23:25:21.683506+0000","last_active":"2024-02-21T23:52:56.423026+0000","last_peered":"2024-02-21T23:52:56.423026+0000","last_clean":"2024-02-21T23:52:56.423026+0000","last_became_active":"2024-02-21T23:25:21.683115+0000","last_became_peered":"2024-02-21T23:25:21.683115+0000","last_unstale":"2024-02-21T23:52:56.423026+0000","last_undegraded":"2024-02-21T23:52:56.423026+0000","last_fullsized":"2024-02-21T23:52:56.423026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:13.865145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":12261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"26'6","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:08.503461+0000","last_change":"2024-02-21T23:25:16.130851+0000","last_active":"2024-02-21T23:53:08.503461+0000","last_peered":"2024-02-21T23:53:08.503461+0000","last_clean":"2024-02-21T23:53:08.503461+0000","last_became_active":"2024-02-21T23:25:16.130604+0000","last_became_peered":"2024-02-21T23:25:16.130604+0000","last_unstale":"2024-02-21T23:53:08.503461+0000","last_undegraded":"2024-02-21T23:53:08.503461+0000","last_fullsized":"2024-02-21T23:53:08.503461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'47","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.495013+0000","last_change":"2024-02-21T23:25:17.192642+0000","last_active":"2024-02-21T23:53:24.495013+0000","last_peered":"2024-02-21T23:53:24.495013+0000","last_clean":"2024-02-21T23:53:24.495013+0000","last_became_active":"2024-02-21T23:25:17.192394+0000","last_became_peered":"2024-02-21T23:25:17.192394+0000","last_unstale":"2024-02-21T23:53:24.495013+0000","last_undegraded":"2024-02-21T23:53:24.495013+0000","last_fullsized":"2024-02-21T23:53:24.495013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'145","reported_seq":174,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:53.583749+0000","last_change":"2024-02-21T23:25:21.679491+0000","last_active":"2024-02-21T23:52:53.583749+0000","last_peered":"2024-02-21T23:52:53.583749+0000","last_clean":"2024-02-21T23:52:53.583749+0000","last_became_active":"2024-02-21T23:25:21.679117+0000","last_became_peered":"2024-02-21T23:25:21.679117+0000","last_unstale":"2024-02-21T23:52:53.583749+0000","last_undegraded":"2024-02-21T23:52:53.583749+0000","last_fullsized":"2024-02-21T23:52:53.583749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:52.680971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":9573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626233+0000","last_change":"2024-02-21T23:25:17.161438+0000","last_active":"2024-02-21T23:53:24.626233+0000","last_peered":"2024-02-21T23:53:24.626233+0000","last_clean":"2024-02-21T23:53:24.626233+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:24.626233+0000","last_undegraded":"2024-02-21T23:53:24.626233+0000","last_fullsized":"2024-02-21T23:53:24.626233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'648","reported_seq":686,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:07.478231+0000","last_change":"2024-02-21T23:25:16.136913+0000","last_active":"2024-02-21T23:53:07.478231+0000","last_peered":"2024-02-21T23:53:07.478231+0000","last_clean":"2024-02-21T23:53:07.478231+0000","last_became_active":"2024-02-21T23:25:16.136628+0000","last_became_peered":"2024-02-21T23:25:16.136628+0000","last_unstale":"2024-02-21T23:53:07.478231+0000","last_undegraded":"2024-02-21T23:53:07.478231+0000","last_fullsized":"2024-02-21T23:53:07.478231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":644,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'134","reported_seq":171,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.695314+0000","last_change":"2024-02-21T23:25:57.526324+0000","last_active":"2024-02-21T23:52:56.695314+0000","last_peered":"2024-02-21T23:52:56.695314+0000","last_clean":"2024-02-21T23:52:56.695314+0000","last_became_active":"2024-02-21T23:25:57.525854+0000","last_became_peered":"2024-02-21T23:25:57.525854+0000","last_unstale":"2024-02-21T23:52:56.695314+0000","last_undegraded":"2024-02-21T23:52:56.695314+0000","last_fullsized":"2024-02-21T23:52:56.695314+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:12:32.356243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.465980+0000","last_change":"2024-02-21T23:25:17.137829+0000","last_active":"2024-02-21T23:53:24.465980+0000","last_peered":"2024-02-21T23:53:24.465980+0000","last_clean":"2024-02-21T23:53:24.465980+0000","last_became_active":"2024-02-21T23:25:17.137557+0000","last_became_peered":"2024-02-21T23:25:17.137557+0000","last_unstale":"2024-02-21T23:53:24.465980+0000","last_undegraded":"2024-02-21T23:53:24.465980+0000","last_fullsized":"2024-02-21T23:53:24.465980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:59.099888+0000","last_change":"2024-02-21T23:25:16.146181+0000","last_active":"2024-02-21T23:52:59.099888+0000","last_peered":"2024-02-21T23:52:59.099888+0000","last_clean":"2024-02-21T23:52:59.099888+0000","last_became_active":"2024-02-21T23:25:16.145932+0000","last_became_peered":"2024-02-21T23:25:16.145932+0000","last_unstale":"2024-02-21T23:52:59.099888+0000","last_undegraded":"2024-02-21T23:52:59.099888+0000","last_fullsized":"2024-02-21T23:52:59.099888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'165","reported_seq":195,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.095405+0000","last_change":"2024-02-21T23:25:21.686044+0000","last_active":"2024-02-21T23:52:55.095405+0000","last_peered":"2024-02-21T23:52:55.095405+0000","last_clean":"2024-02-21T23:52:55.095405+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:52:55.095405+0000","last_undegraded":"2024-02-21T23:52:55.095405+0000","last_fullsized":"2024-02-21T23:52:55.095405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:39.740454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'37","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471561+0000","last_change":"2024-02-21T23:25:17.190236+0000","last_active":"2024-02-21T23:53:24.471561+0000","last_peered":"2024-02-21T23:53:24.471561+0000","last_clean":"2024-02-21T23:53:24.471561+0000","last_became_active":"2024-02-21T23:25:17.189957+0000","last_became_peered":"2024-02-21T23:25:17.189957+0000","last_unstale":"2024-02-21T23:53:24.471561+0000","last_undegraded":"2024-02-21T23:53:24.471561+0000","last_fullsized":"2024-02-21T23:53:24.471561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:03.326866+0000","last_change":"2024-02-21T23:25:16.148108+0000","last_active":"2024-02-21T23:53:03.326866+0000","last_peered":"2024-02-21T23:53:03.326866+0000","last_clean":"2024-02-21T23:53:03.326866+0000","last_became_active":"2024-02-21T23:25:16.147930+0000","last_became_peered":"2024-02-21T23:25:16.147930+0000","last_unstale":"2024-02-21T23:53:03.326866+0000","last_undegraded":"2024-02-21T23:53:03.326866+0000","last_fullsized":"2024-02-21T23:53:03.326866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'132","reported_seq":162,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.410926+0000","last_change":"2024-02-21T23:25:22.248150+0000","last_active":"2024-02-21T23:52:56.410926+0000","last_peered":"2024-02-21T23:52:56.410926+0000","last_clean":"2024-02-21T23:52:56.410926+0000","last_became_active":"2024-02-21T23:25:22.247769+0000","last_became_peered":"2024-02-21T23:25:22.247769+0000","last_unstale":"2024-02-21T23:52:56.410926+0000","last_undegraded":"2024-02-21T23:52:56.410926+0000","last_fullsized":"2024-02-21T23:52:56.410926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"26'34","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.477731+0000","last_change":"2024-02-21T23:25:17.189758+0000","last_active":"2024-02-21T23:53:24.477731+0000","last_peered":"2024-02-21T23:53:24.477731+0000","last_clean":"2024-02-21T23:53:24.477731+0000","last_became_active":"2024-02-21T23:25:17.189492+0000","last_became_peered":"2024-02-21T23:25:17.189492+0000","last_unstale":"2024-02-21T23:53:24.477731+0000","last_undegraded":"2024-02-21T23:53:24.477731+0000","last_fullsized":"2024-02-21T23:53:24.477731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:42.526598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'7","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:17.858179+0000","last_change":"2024-02-21T23:25:16.131611+0000","last_active":"2024-02-21T23:53:17.858179+0000","last_peered":"2024-02-21T23:53:17.858179+0000","last_clean":"2024-02-21T23:53:17.858179+0000","last_became_active":"2024-02-21T23:25:16.131158+0000","last_became_peered":"2024-02-21T23:25:16.131158+0000","last_unstale":"2024-02-21T23:53:17.858179+0000","last_undegraded":"2024-02-21T23:53:17.858179+0000","last_fullsized":"2024-02-21T23:53:17.858179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'116","reported_seq":146,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.014642+0000","last_change":"2024-02-21T23:25:21.683821+0000","last_active":"2024-02-21T23:52:52.014642+0000","last_peered":"2024-02-21T23:52:52.014642+0000","last_clean":"2024-02-21T23:52:52.014642+0000","last_became_active":"2024-02-21T23:25:21.683466+0000","last_became_peered":"2024-02-21T23:25:21.683466+0000","last_unstale":"2024-02-21T23:52:52.014642+0000","last_undegraded":"2024-02-21T23:52:52.014642+0000","last_fullsized":"2024-02-21T23:52:52.014642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":7338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"26'38","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.602687+0000","last_change":"2024-02-21T23:25:17.191394+0000","last_active":"2024-02-21T23:53:24.602687+0000","last_peered":"2024-02-21T23:53:24.602687+0000","last_clean":"2024-02-21T23:53:24.602687+0000","last_became_active":"2024-02-21T23:25:17.191106+0000","last_became_peered":"2024-02-21T23:25:17.191106+0000","last_unstale":"2024-02-21T23:53:24.602687+0000","last_undegraded":"2024-02-21T23:53:24.602687+0000","last_fullsized":"2024-02-21T23:53:24.602687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'9","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.843551+0000","last_change":"2024-02-21T23:25:16.134862+0000","last_active":"2024-02-21T23:53:18.843551+0000","last_peered":"2024-02-21T23:53:18.843551+0000","last_clean":"2024-02-21T23:53:18.843551+0000","last_became_active":"2024-02-21T23:25:16.134602+0000","last_became_peered":"2024-02-21T23:25:16.134602+0000","last_unstale":"2024-02-21T23:53:18.843551+0000","last_undegraded":"2024-02-21T23:53:18.843551+0000","last_fullsized":"2024-02-21T23:53:18.843551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'143","reported_seq":174,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.422758+0000","last_change":"2024-02-21T23:25:22.249606+0000","last_active":"2024-02-21T23:52:56.422758+0000","last_peered":"2024-02-21T23:52:56.422758+0000","last_clean":"2024-02-21T23:52:56.422758+0000","last_became_active":"2024-02-21T23:25:22.249214+0000","last_became_peered":"2024-02-21T23:25:22.249214+0000","last_unstale":"2024-02-21T23:52:56.422758+0000","last_undegraded":"2024-02-21T23:52:56.422758+0000","last_fullsized":"2024-02-21T23:52:56.422758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:36.929076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":6395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'33","reported_seq":70,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471655+0000","last_change":"2024-02-21T23:25:17.183290+0000","last_active":"2024-02-21T23:53:24.471655+0000","last_peered":"2024-02-21T23:53:24.471655+0000","last_clean":"2024-02-21T23:53:24.471655+0000","last_became_active":"2024-02-21T23:25:17.183047+0000","last_became_peered":"2024-02-21T23:25:17.183047+0000","last_unstale":"2024-02-21T23:53:24.471655+0000","last_undegraded":"2024-02-21T23:53:24.471655+0000","last_fullsized":"2024-02-21T23:53:24.471655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'1368","reported_seq":1407,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.000230+0000","last_change":"2024-02-21T23:25:16.146202+0000","last_active":"2024-02-21T23:53:23.000230+0000","last_peered":"2024-02-21T23:53:23.000230+0000","last_clean":"2024-02-21T23:53:23.000230+0000","last_became_active":"2024-02-21T23:25:16.145911+0000","last_became_peered":"2024-02-21T23:25:16.145911+0000","last_unstale":"2024-02-21T23:53:23.000230+0000","last_undegraded":"2024-02-21T23:53:23.000230+0000","last_fullsized":"2024-02-21T23:53:23.000230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3511,"num_write_kb":1242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'135","reported_seq":167,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700303+0000","last_change":"2024-02-21T23:25:22.243297+0000","last_active":"2024-02-21T23:52:56.700303+0000","last_peered":"2024-02-21T23:52:56.700303+0000","last_clean":"2024-02-21T23:52:56.700303+0000","last_became_active":"2024-02-21T23:25:22.242922+0000","last_became_peered":"2024-02-21T23:25:22.242922+0000","last_unstale":"2024-02-21T23:52:56.700303+0000","last_undegraded":"2024-02-21T23:52:56.700303+0000","last_fullsized":"2024-02-21T23:52:56.700303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:52:51.773574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":7447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"26'37","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.456470+0000","last_change":"2024-02-21T23:25:17.190577+0000","last_active":"2024-02-21T23:53:24.456470+0000","last_peered":"2024-02-21T23:53:24.456470+0000","last_clean":"2024-02-21T23:53:24.456470+0000","last_became_active":"2024-02-21T23:25:17.190332+0000","last_became_peered":"2024-02-21T23:25:17.190332+0000","last_unstale":"2024-02-21T23:53:24.456470+0000","last_undegraded":"2024-02-21T23:53:24.456470+0000","last_fullsized":"2024-02-21T23:53:24.456470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'3","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.853192+0000","last_change":"2024-02-21T23:25:16.134076+0000","last_active":"2024-02-21T23:53:15.853192+0000","last_peered":"2024-02-21T23:53:15.853192+0000","last_clean":"2024-02-21T23:53:15.853192+0000","last_became_active":"2024-02-21T23:25:16.133816+0000","last_became_peered":"2024-02-21T23:25:16.133816+0000","last_unstale":"2024-02-21T23:53:15.853192+0000","last_undegraded":"2024-02-21T23:53:15.853192+0000","last_fullsized":"2024-02-21T23:53:15.853192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'127","reported_seq":158,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.466825+0000","last_change":"2024-02-21T23:25:22.243360+0000","last_active":"2024-02-21T23:52:48.466825+0000","last_peered":"2024-02-21T23:52:48.466825+0000","last_clean":"2024-02-21T23:52:48.466825+0000","last_became_active":"2024-02-21T23:25:22.242958+0000","last_became_peered":"2024-02-21T23:25:22.242958+0000","last_unstale":"2024-02-21T23:52:48.466825+0000","last_undegraded":"2024-02-21T23:52:48.466825+0000","last_fullsized":"2024-02-21T23:52:48.466825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:43.992101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":7411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494810+0000","last_change":"2024-02-21T23:25:17.192402+0000","last_active":"2024-02-21T23:53:24.494810+0000","last_peered":"2024-02-21T23:53:24.494810+0000","last_clean":"2024-02-21T23:53:24.494810+0000","last_became_active":"2024-02-21T23:25:17.192138+0000","last_became_peered":"2024-02-21T23:25:17.192138+0000","last_unstale":"2024-02-21T23:53:24.494810+0000","last_undegraded":"2024-02-21T23:53:24.494810+0000","last_fullsized":"2024-02-21T23:53:24.494810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:03.326742+0000","last_change":"2024-02-21T23:25:16.145521+0000","last_active":"2024-02-21T23:53:03.326742+0000","last_peered":"2024-02-21T23:53:03.326742+0000","last_clean":"2024-02-21T23:53:03.326742+0000","last_became_active":"2024-02-21T23:25:16.145255+0000","last_became_peered":"2024-02-21T23:25:16.145255+0000","last_unstale":"2024-02-21T23:53:03.326742+0000","last_undegraded":"2024-02-21T23:53:03.326742+0000","last_fullsized":"2024-02-21T23:53:03.326742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'208","reported_seq":239,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.413727+0000","last_change":"2024-02-21T23:25:21.683849+0000","last_active":"2024-02-21T23:52:56.413727+0000","last_peered":"2024-02-21T23:52:56.413727+0000","last_clean":"2024-02-21T23:52:56.413727+0000","last_became_active":"2024-02-21T23:25:21.683379+0000","last_became_peered":"2024-02-21T23:25:21.683379+0000","last_unstale":"2024-02-21T23:52:56.413727+0000","last_undegraded":"2024-02-21T23:52:56.413727+0000","last_fullsized":"2024-02-21T23:52:56.413727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":191,"num_write_kb":16487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"26'31","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.482603+0000","last_change":"2024-02-21T23:25:17.166232+0000","last_active":"2024-02-21T23:53:24.482603+0000","last_peered":"2024-02-21T23:53:24.482603+0000","last_clean":"2024-02-21T23:53:24.482603+0000","last_became_active":"2024-02-21T23:25:17.165999+0000","last_became_peered":"2024-02-21T23:25:17.165999+0000","last_unstale":"2024-02-21T23:53:24.482603+0000","last_undegraded":"2024-02-21T23:53:24.482603+0000","last_fullsized":"2024-02-21T23:53:24.482603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:45.125986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:23.691688+0000","last_change":"2024-02-21T23:25:16.147082+0000","last_active":"2024-02-21T23:52:23.691688+0000","last_peered":"2024-02-21T23:52:23.691688+0000","last_clean":"2024-02-21T23:52:23.691688+0000","last_became_active":"2024-02-21T23:25:16.146797+0000","last_became_peered":"2024-02-21T23:25:16.146797+0000","last_unstale":"2024-02-21T23:52:23.691688+0000","last_undegraded":"2024-02-21T23:52:23.691688+0000","last_fullsized":"2024-02-21T23:52:23.691688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'125","reported_seq":150,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.407469+0000","last_change":"2024-02-21T23:25:57.583768+0000","last_active":"2024-02-21T23:52:56.407469+0000","last_peered":"2024-02-21T23:52:56.407469+0000","last_clean":"2024-02-21T23:52:56.407469+0000","last_became_active":"2024-02-21T23:25:57.583297+0000","last_became_peered":"2024-02-21T23:25:57.583297+0000","last_unstale":"2024-02-21T23:52:56.407469+0000","last_undegraded":"2024-02-21T23:52:56.407469+0000","last_fullsized":"2024-02-21T23:52:56.407469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:19:14.798657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":6113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,6],"acting":[0,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"26'33","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.454823+0000","last_change":"2024-02-21T23:25:17.190425+0000","last_active":"2024-02-21T23:53:24.454823+0000","last_peered":"2024-02-21T23:53:24.454823+0000","last_clean":"2024-02-21T23:53:24.454823+0000","last_became_active":"2024-02-21T23:25:17.190172+0000","last_became_peered":"2024-02-21T23:25:17.190172+0000","last_unstale":"2024-02-21T23:53:24.454823+0000","last_undegraded":"2024-02-21T23:53:24.454823+0000","last_fullsized":"2024-02-21T23:53:24.454823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'999","reported_seq":2032,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.855182+0000","last_change":"2024-02-21T23:25:16.108263+0000","last_active":"2024-02-21T23:53:23.855182+0000","last_peered":"2024-02-21T23:53:23.855182+0000","last_clean":"2024-02-21T23:53:23.855182+0000","last_became_active":"2024-02-21T23:25:16.107822+0000","last_became_peered":"2024-02-21T23:25:16.107822+0000","last_unstale":"2024-02-21T23:53:23.855182+0000","last_undegraded":"2024-02-21T23:53:23.855182+0000","last_fullsized":"2024-02-21T23:53:23.855182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":995,"num_write_kb":1061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'134","reported_seq":164,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.702471+0000","last_change":"2024-02-21T23:25:21.686034+0000","last_active":"2024-02-21T23:52:56.702471+0000","last_peered":"2024-02-21T23:52:56.702471+0000","last_clean":"2024-02-21T23:52:56.702471+0000","last_became_active":"2024-02-21T23:25:21.685546+0000","last_became_peered":"2024-02-21T23:25:21.685546+0000","last_unstale":"2024-02-21T23:52:56.702471+0000","last_undegraded":"2024-02-21T23:52:56.702471+0000","last_fullsized":"2024-02-21T23:52:56.702471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":10678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607087+0000","last_change":"2024-02-21T23:25:17.161738+0000","last_active":"2024-02-21T23:53:24.607087+0000","last_peered":"2024-02-21T23:53:24.607087+0000","last_clean":"2024-02-21T23:53:24.607087+0000","last_became_active":"2024-02-21T23:25:17.161459+0000","last_became_peered":"2024-02-21T23:25:17.161459+0000","last_unstale":"2024-02-21T23:53:24.607087+0000","last_undegraded":"2024-02-21T23:53:24.607087+0000","last_fullsized":"2024-02-21T23:53:24.607087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:31:58.434561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'541","reported_seq":581,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.846835+0000","last_change":"2024-02-21T23:25:16.147162+0000","last_active":"2024-02-21T23:53:18.846835+0000","last_peered":"2024-02-21T23:53:18.846835+0000","last_clean":"2024-02-21T23:53:18.846835+0000","last_became_active":"2024-02-21T23:25:16.146850+0000","last_became_peered":"2024-02-21T23:25:16.146850+0000","last_unstale":"2024-02-21T23:53:18.846835+0000","last_undegraded":"2024-02-21T23:53:18.846835+0000","last_fullsized":"2024-02-21T23:53:18.846835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'140","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.987450+0000","last_change":"2024-02-21T23:25:22.248109+0000","last_active":"2024-02-21T23:52:52.987450+0000","last_peered":"2024-02-21T23:52:52.987450+0000","last_clean":"2024-02-21T23:52:52.987450+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:52:52.987450+0000","last_undegraded":"2024-02-21T23:52:52.987450+0000","last_fullsized":"2024-02-21T23:52:52.987450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:36:18.099325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"26'35","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.466335+0000","last_change":"2024-02-21T23:25:57.600968+0000","last_active":"2024-02-21T23:53:24.466335+0000","last_peered":"2024-02-21T23:53:24.466335+0000","last_clean":"2024-02-21T23:53:24.466335+0000","last_became_active":"2024-02-21T23:25:57.600644+0000","last_became_peered":"2024-02-21T23:25:57.600644+0000","last_unstale":"2024-02-21T23:53:24.466335+0000","last_undegraded":"2024-02-21T23:53:24.466335+0000","last_fullsized":"2024-02-21T23:53:24.466335+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:13:09.640854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'5","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:10.587650+0000","last_change":"2024-02-21T23:25:16.127688+0000","last_active":"2024-02-21T23:53:10.587650+0000","last_peered":"2024-02-21T23:53:10.587650+0000","last_clean":"2024-02-21T23:53:10.587650+0000","last_became_active":"2024-02-21T23:25:16.127388+0000","last_became_peered":"2024-02-21T23:25:16.127388+0000","last_unstale":"2024-02-21T23:53:10.587650+0000","last_undegraded":"2024-02-21T23:53:10.587650+0000","last_fullsized":"2024-02-21T23:53:10.587650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:51:13.500334+0000","last_change":"2024-02-21T23:25:05.612817+0000","last_active":"2024-02-21T23:51:13.500334+0000","last_peered":"2024-02-21T23:51:13.500334+0000","last_clean":"2024-02-21T23:51:13.500334+0000","last_became_active":"2024-02-21T23:25:05.612483+0000","last_became_peered":"2024-02-21T23:25:05.612483+0000","last_unstale":"2024-02-21T23:51:13.500334+0000","last_undegraded":"2024-02-21T23:51:13.500334+0000","last_fullsized":"2024-02-21T23:51:13.500334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_clean_scrub_stamp":"2024-02-21T23:25:03.736732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:23:33.073040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'168","reported_seq":199,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.098108+0000","last_change":"2024-02-21T23:25:22.248174+0000","last_active":"2024-02-21T23:52:55.098108+0000","last_peered":"2024-02-21T23:52:55.098108+0000","last_clean":"2024-02-21T23:52:55.098108+0000","last_became_active":"2024-02-21T23:25:22.247834+0000","last_became_peered":"2024-02-21T23:25:22.247834+0000","last_unstale":"2024-02-21T23:52:55.098108+0000","last_undegraded":"2024-02-21T23:52:55.098108+0000","last_fullsized":"2024-02-21T23:52:55.098108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:13.953358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":10993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604646+0000","last_change":"2024-02-21T23:25:17.192575+0000","last_active":"2024-02-21T23:53:24.604646+0000","last_peered":"2024-02-21T23:53:24.604646+0000","last_clean":"2024-02-21T23:53:24.604646+0000","last_became_active":"2024-02-21T23:25:17.192293+0000","last_became_peered":"2024-02-21T23:25:17.192293+0000","last_unstale":"2024-02-21T23:53:24.604646+0000","last_undegraded":"2024-02-21T23:53:24.604646+0000","last_fullsized":"2024-02-21T23:53:24.604646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:57.046912+0000","last_change":"2024-02-21T23:25:16.128369+0000","last_active":"2024-02-21T23:52:57.046912+0000","last_peered":"2024-02-21T23:52:57.046912+0000","last_clean":"2024-02-21T23:52:57.046912+0000","last_became_active":"2024-02-21T23:25:16.128067+0000","last_became_peered":"2024-02-21T23:25:16.128067+0000","last_unstale":"2024-02-21T23:52:57.046912+0000","last_undegraded":"2024-02-21T23:52:57.046912+0000","last_fullsized":"2024-02-21T23:52:57.046912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:22.671171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'138","reported_seq":168,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.670157+0000","last_change":"2024-02-21T23:25:22.243282+0000","last_active":"2024-02-21T23:52:52.670157+0000","last_peered":"2024-02-21T23:52:52.670157+0000","last_clean":"2024-02-21T23:52:52.670157+0000","last_became_active":"2024-02-21T23:25:22.242916+0000","last_became_peered":"2024-02-21T23:25:22.242916+0000","last_unstale":"2024-02-21T23:52:52.670157+0000","last_undegraded":"2024-02-21T23:52:52.670157+0000","last_fullsized":"2024-02-21T23:52:52.670157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"26'40","reported_seq":76,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493486+0000","last_change":"2024-02-21T23:25:17.185418+0000","last_active":"2024-02-21T23:53:24.493486+0000","last_peered":"2024-02-21T23:53:24.493486+0000","last_clean":"2024-02-21T23:53:24.493486+0000","last_became_active":"2024-02-21T23:25:17.185119+0000","last_became_peered":"2024-02-21T23:25:17.185119+0000","last_unstale":"2024-02-21T23:53:24.493486+0000","last_undegraded":"2024-02-21T23:53:24.493486+0000","last_fullsized":"2024-02-21T23:53:24.493486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:18.471011+0000","last_change":"2024-02-21T23:25:16.151088+0000","last_active":"2024-02-21T23:52:18.471011+0000","last_peered":"2024-02-21T23:52:18.471011+0000","last_clean":"2024-02-21T23:52:18.471011+0000","last_became_active":"2024-02-21T23:25:16.150819+0000","last_became_peered":"2024-02-21T23:25:16.150819+0000","last_unstale":"2024-02-21T23:52:18.471011+0000","last_undegraded":"2024-02-21T23:52:18.471011+0000","last_fullsized":"2024-02-21T23:52:18.471011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'189","reported_seq":219,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.685417+0000","last_change":"2024-02-21T23:25:21.684592+0000","last_active":"2024-02-21T23:52:56.685417+0000","last_peered":"2024-02-21T23:52:56.685417+0000","last_clean":"2024-02-21T23:52:56.685417+0000","last_became_active":"2024-02-21T23:25:21.684192+0000","last_became_peered":"2024-02-21T23:25:21.684192+0000","last_unstale":"2024-02-21T23:52:56.685417+0000","last_undegraded":"2024-02-21T23:52:56.685417+0000","last_fullsized":"2024-02-21T23:52:56.685417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:51.734754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":12683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.606181+0000","last_change":"2024-02-21T23:25:17.184268+0000","last_active":"2024-02-21T23:53:24.606181+0000","last_peered":"2024-02-21T23:53:24.606181+0000","last_clean":"2024-02-21T23:53:24.606181+0000","last_became_active":"2024-02-21T23:25:17.184027+0000","last_became_peered":"2024-02-21T23:25:17.184027+0000","last_unstale":"2024-02-21T23:53:24.606181+0000","last_undegraded":"2024-02-21T23:53:24.606181+0000","last_fullsized":"2024-02-21T23:53:24.606181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'4","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:09.554814+0000","last_change":"2024-02-21T23:25:16.108099+0000","last_active":"2024-02-21T23:53:09.554814+0000","last_peered":"2024-02-21T23:53:09.554814+0000","last_clean":"2024-02-21T23:53:09.554814+0000","last_became_active":"2024-02-21T23:25:16.107815+0000","last_became_peered":"2024-02-21T23:25:16.107815+0000","last_unstale":"2024-02-21T23:53:09.554814+0000","last_undegraded":"2024-02-21T23:53:09.554814+0000","last_fullsized":"2024-02-21T23:53:09.554814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'131","reported_seq":161,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.464577+0000","last_change":"2024-02-21T23:25:22.248926+0000","last_active":"2024-02-21T23:52:48.464577+0000","last_peered":"2024-02-21T23:52:48.464577+0000","last_clean":"2024-02-21T23:52:48.464577+0000","last_became_active":"2024-02-21T23:25:22.248558+0000","last_became_peered":"2024-02-21T23:25:22.248558+0000","last_unstale":"2024-02-21T23:52:48.464577+0000","last_undegraded":"2024-02-21T23:52:48.464577+0000","last_fullsized":"2024-02-21T23:52:48.464577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"26'23","reported_seq":59,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605888+0000","last_change":"2024-02-21T23:25:17.183254+0000","last_active":"2024-02-21T23:53:24.605888+0000","last_peered":"2024-02-21T23:53:24.605888+0000","last_clean":"2024-02-21T23:53:24.605888+0000","last_became_active":"2024-02-21T23:25:17.183020+0000","last_became_peered":"2024-02-21T23:25:17.183020+0000","last_unstale":"2024-02-21T23:53:24.605888+0000","last_undegraded":"2024-02-21T23:53:24.605888+0000","last_fullsized":"2024-02-21T23:53:24.605888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:23.581289+0000","last_change":"2024-02-21T23:25:16.129131+0000","last_active":"2024-02-21T23:52:23.581289+0000","last_peered":"2024-02-21T23:52:23.581289+0000","last_clean":"2024-02-21T23:52:23.581289+0000","last_became_active":"2024-02-21T23:25:16.128853+0000","last_became_peered":"2024-02-21T23:25:16.128853+0000","last_unstale":"2024-02-21T23:52:23.581289+0000","last_undegraded":"2024-02-21T23:52:23.581289+0000","last_fullsized":"2024-02-21T23:52:23.581289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:29.302181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'163","reported_seq":193,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.422752+0000","last_change":"2024-02-21T23:25:21.684342+0000","last_active":"2024-02-21T23:52:56.422752+0000","last_peered":"2024-02-21T23:52:56.422752+0000","last_clean":"2024-02-21T23:52:56.422752+0000","last_became_active":"2024-02-21T23:25:21.683991+0000","last_became_peered":"2024-02-21T23:25:21.683991+0000","last_unstale":"2024-02-21T23:52:56.422752+0000","last_undegraded":"2024-02-21T23:52:56.422752+0000","last_fullsized":"2024-02-21T23:52:56.422752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:21:00.312314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":12715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"26'37","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.606217+0000","last_change":"2024-02-21T23:25:17.183442+0000","last_active":"2024-02-21T23:53:24.606217+0000","last_peered":"2024-02-21T23:53:24.606217+0000","last_clean":"2024-02-21T23:53:24.606217+0000","last_became_active":"2024-02-21T23:25:17.183057+0000","last_became_peered":"2024-02-21T23:25:17.183057+0000","last_unstale":"2024-02-21T23:53:24.606217+0000","last_undegraded":"2024-02-21T23:53:24.606217+0000","last_fullsized":"2024-02-21T23:53:24.606217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'13","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.623678+0000","last_change":"2024-02-21T23:25:16.145801+0000","last_active":"2024-02-21T23:53:18.623678+0000","last_peered":"2024-02-21T23:53:18.623678+0000","last_clean":"2024-02-21T23:53:18.623678+0000","last_became_active":"2024-02-21T23:25:16.145523+0000","last_became_peered":"2024-02-21T23:25:16.145523+0000","last_unstale":"2024-02-21T23:53:18.623678+0000","last_undegraded":"2024-02-21T23:53:18.623678+0000","last_fullsized":"2024-02-21T23:53:18.623678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'157","reported_seq":186,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.093033+0000","last_change":"2024-02-21T23:25:21.679191+0000","last_active":"2024-02-21T23:52:55.093033+0000","last_peered":"2024-02-21T23:52:55.093033+0000","last_clean":"2024-02-21T23:52:55.093033+0000","last_became_active":"2024-02-21T23:25:21.678715+0000","last_became_peered":"2024-02-21T23:25:21.678715+0000","last_unstale":"2024-02-21T23:52:55.093033+0000","last_undegraded":"2024-02-21T23:52:55.093033+0000","last_fullsized":"2024-02-21T23:52:55.093033+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":11459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"26'29","reported_seq":65,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.625866+0000","last_change":"2024-02-21T23:25:17.138401+0000","last_active":"2024-02-21T23:53:24.625866+0000","last_peered":"2024-02-21T23:53:24.625866+0000","last_clean":"2024-02-21T23:53:24.625866+0000","last_became_active":"2024-02-21T23:25:17.138136+0000","last_became_peered":"2024-02-21T23:25:17.138136+0000","last_unstale":"2024-02-21T23:53:24.625866+0000","last_undegraded":"2024-02-21T23:53:24.625866+0000","last_fullsized":"2024-02-21T23:53:24.625866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'8","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.890247+0000","last_change":"2024-02-21T23:25:16.130209+0000","last_active":"2024-02-21T23:53:15.890247+0000","last_peered":"2024-02-21T23:53:15.890247+0000","last_clean":"2024-02-21T23:53:15.890247+0000","last_became_active":"2024-02-21T23:25:16.129974+0000","last_became_peered":"2024-02-21T23:25:16.129974+0000","last_unstale":"2024-02-21T23:53:15.890247+0000","last_undegraded":"2024-02-21T23:53:15.890247+0000","last_fullsized":"2024-02-21T23:53:15.890247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'138","reported_seq":167,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700347+0000","last_change":"2024-02-21T23:25:22.243303+0000","last_active":"2024-02-21T23:52:56.700347+0000","last_peered":"2024-02-21T23:52:56.700347+0000","last_clean":"2024-02-21T23:52:56.700347+0000","last_became_active":"2024-02-21T23:25:22.242960+0000","last_became_peered":"2024-02-21T23:25:22.242960+0000","last_unstale":"2024-02-21T23:52:56.700347+0000","last_undegraded":"2024-02-21T23:52:56.700347+0000","last_fullsized":"2024-02-21T23:52:56.700347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:54:16.386354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":8319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"26'48","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605915+0000","last_change":"2024-02-21T23:25:17.184233+0000","last_active":"2024-02-21T23:53:24.605915+0000","last_peered":"2024-02-21T23:53:24.605915+0000","last_clean":"2024-02-21T23:53:24.605915+0000","last_became_active":"2024-02-21T23:25:17.183980+0000","last_became_peered":"2024-02-21T23:25:17.183980+0000","last_unstale":"2024-02-21T23:53:24.605915+0000","last_undegraded":"2024-02-21T23:53:24.605915+0000","last_fullsized":"2024-02-21T23:53:24.605915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'3","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:14.807121+0000","last_change":"2024-02-21T23:25:16.131147+0000","last_active":"2024-02-21T23:53:14.807121+0000","last_peered":"2024-02-21T23:53:14.807121+0000","last_clean":"2024-02-21T23:53:14.807121+0000","last_became_active":"2024-02-21T23:25:16.130553+0000","last_became_peered":"2024-02-21T23:25:16.130553+0000","last_unstale":"2024-02-21T23:53:14.807121+0000","last_undegraded":"2024-02-21T23:53:14.807121+0000","last_fullsized":"2024-02-21T23:53:14.807121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'97","reported_seq":127,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.566724+0000","last_change":"2024-02-21T23:25:22.243261+0000","last_active":"2024-02-21T23:52:56.566724+0000","last_peered":"2024-02-21T23:52:56.566724+0000","last_clean":"2024-02-21T23:52:56.566724+0000","last_became_active":"2024-02-21T23:25:22.242900+0000","last_became_peered":"2024-02-21T23:25:22.242900+0000","last_unstale":"2024-02-21T23:52:56.566724+0000","last_undegraded":"2024-02-21T23:52:56.566724+0000","last_fullsized":"2024-02-21T23:52:56.566724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:34:44.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":7038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'44","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607089+0000","last_change":"2024-02-21T23:25:17.183303+0000","last_active":"2024-02-21T23:53:24.607089+0000","last_peered":"2024-02-21T23:53:24.607089+0000","last_clean":"2024-02-21T23:53:24.607089+0000","last_became_active":"2024-02-21T23:25:17.183042+0000","last_became_peered":"2024-02-21T23:25:17.183042+0000","last_unstale":"2024-02-21T23:53:24.607089+0000","last_undegraded":"2024-02-21T23:53:24.607089+0000","last_fullsized":"2024-02-21T23:53:24.607089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'251","reported_seq":289,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.291094+0000","last_change":"2024-02-21T23:25:16.127609+0000","last_active":"2024-02-21T23:53:24.291094+0000","last_peered":"2024-02-21T23:53:24.291094+0000","last_clean":"2024-02-21T23:53:24.291094+0000","last_became_active":"2024-02-21T23:25:16.127375+0000","last_became_peered":"2024-02-21T23:25:16.127375+0000","last_unstale":"2024-02-21T23:53:24.291094+0000","last_undegraded":"2024-02-21T23:53:24.291094+0000","last_fullsized":"2024-02-21T23:53:24.291094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2561517,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":2649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'148","reported_seq":177,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.755700+0000","last_change":"2024-02-21T23:25:22.248329+0000","last_active":"2024-02-21T23:52:56.755700+0000","last_peered":"2024-02-21T23:52:56.755700+0000","last_clean":"2024-02-21T23:52:56.755700+0000","last_became_active":"2024-02-21T23:25:22.247889+0000","last_became_peered":"2024-02-21T23:25:22.247889+0000","last_unstale":"2024-02-21T23:52:56.755700+0000","last_undegraded":"2024-02-21T23:52:56.755700+0000","last_fullsized":"2024-02-21T23:52:56.755700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":13204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"26'37","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.527489+0000","last_change":"2024-02-21T23:25:57.527196+0000","last_active":"2024-02-21T23:53:24.527489+0000","last_peered":"2024-02-21T23:53:24.527489+0000","last_clean":"2024-02-21T23:53:24.527489+0000","last_became_active":"2024-02-21T23:25:57.526874+0000","last_became_peered":"2024-02-21T23:25:57.526874+0000","last_unstale":"2024-02-21T23:53:24.527489+0000","last_undegraded":"2024-02-21T23:53:24.527489+0000","last_fullsized":"2024-02-21T23:53:24.527489+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"26'835","reported_seq":873,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:16.019825+0000","last_change":"2024-02-21T23:25:16.116460+0000","last_active":"2024-02-21T23:53:16.019825+0000","last_peered":"2024-02-21T23:53:16.019825+0000","last_clean":"2024-02-21T23:53:16.019825+0000","last_became_active":"2024-02-21T23:25:16.116162+0000","last_became_peered":"2024-02-21T23:25:16.116162+0000","last_unstale":"2024-02-21T23:53:16.019825+0000","last_undegraded":"2024-02-21T23:53:16.019825+0000","last_fullsized":"2024-02-21T23:53:16.019825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'155","reported_seq":185,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.702447+0000","last_change":"2024-02-21T23:25:21.683199+0000","last_active":"2024-02-21T23:52:56.702447+0000","last_peered":"2024-02-21T23:52:56.702447+0000","last_clean":"2024-02-21T23:52:56.702447+0000","last_became_active":"2024-02-21T23:25:21.682425+0000","last_became_peered":"2024-02-21T23:25:21.682425+0000","last_unstale":"2024-02-21T23:52:56.702447+0000","last_undegraded":"2024-02-21T23:52:56.702447+0000","last_fullsized":"2024-02-21T23:52:56.702447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:47.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":9421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'32","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.487783+0000","last_change":"2024-02-21T23:25:17.184837+0000","last_active":"2024-02-21T23:53:24.487783+0000","last_peered":"2024-02-21T23:53:24.487783+0000","last_clean":"2024-02-21T23:53:24.487783+0000","last_became_active":"2024-02-21T23:25:17.184574+0000","last_became_peered":"2024-02-21T23:25:17.184574+0000","last_unstale":"2024-02-21T23:53:24.487783+0000","last_undegraded":"2024-02-21T23:53:24.487783+0000","last_fullsized":"2024-02-21T23:53:24.487783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'697","reported_seq":734,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.851348+0000","last_change":"2024-02-21T23:25:16.151031+0000","last_active":"2024-02-21T23:53:18.851348+0000","last_peered":"2024-02-21T23:53:18.851348+0000","last_clean":"2024-02-21T23:53:18.851348+0000","last_became_active":"2024-02-21T23:25:16.150767+0000","last_became_peered":"2024-02-21T23:25:16.150767+0000","last_unstale":"2024-02-21T23:53:18.851348+0000","last_undegraded":"2024-02-21T23:53:18.851348+0000","last_fullsized":"2024-02-21T23:53:18.851348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'152","reported_seq":182,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.422949+0000","last_change":"2024-02-21T23:25:21.683995+0000","last_active":"2024-02-21T23:52:56.422949+0000","last_peered":"2024-02-21T23:52:56.422949+0000","last_clean":"2024-02-21T23:52:56.422949+0000","last_became_active":"2024-02-21T23:25:21.683415+0000","last_became_peered":"2024-02-21T23:25:21.683415+0000","last_unstale":"2024-02-21T23:52:56.422949+0000","last_undegraded":"2024-02-21T23:52:56.422949+0000","last_fullsized":"2024-02-21T23:52:56.422949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"26'42","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604682+0000","last_change":"2024-02-21T23:25:17.187230+0000","last_active":"2024-02-21T23:53:24.604682+0000","last_peered":"2024-02-21T23:53:24.604682+0000","last_clean":"2024-02-21T23:53:24.604682+0000","last_became_active":"2024-02-21T23:25:17.186952+0000","last_became_peered":"2024-02-21T23:25:17.186952+0000","last_unstale":"2024-02-21T23:53:24.604682+0000","last_undegraded":"2024-02-21T23:53:24.604682+0000","last_fullsized":"2024-02-21T23:53:24.604682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:00.113126+0000","last_change":"2024-02-21T23:25:16.134722+0000","last_active":"2024-02-21T23:53:00.113126+0000","last_peered":"2024-02-21T23:53:00.113126+0000","last_clean":"2024-02-21T23:53:00.113126+0000","last_became_active":"2024-02-21T23:25:16.134458+0000","last_became_peered":"2024-02-21T23:25:16.134458+0000","last_unstale":"2024-02-21T23:53:00.113126+0000","last_undegraded":"2024-02-21T23:53:00.113126+0000","last_fullsized":"2024-02-21T23:53:00.113126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'139","reported_seq":169,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700599+0000","last_change":"2024-02-21T23:25:21.684109+0000","last_active":"2024-02-21T23:52:56.700599+0000","last_peered":"2024-02-21T23:52:56.700599+0000","last_clean":"2024-02-21T23:52:56.700599+0000","last_became_active":"2024-02-21T23:25:21.683705+0000","last_became_peered":"2024-02-21T23:25:21.683705+0000","last_unstale":"2024-02-21T23:52:56.700599+0000","last_undegraded":"2024-02-21T23:52:56.700599+0000","last_fullsized":"2024-02-21T23:52:56.700599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":10133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"26'42","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493527+0000","last_change":"2024-02-21T23:25:17.188374+0000","last_active":"2024-02-21T23:53:24.493527+0000","last_peered":"2024-02-21T23:53:24.493527+0000","last_clean":"2024-02-21T23:53:24.493527+0000","last_became_active":"2024-02-21T23:25:17.187792+0000","last_became_peered":"2024-02-21T23:25:17.187792+0000","last_unstale":"2024-02-21T23:53:24.493527+0000","last_undegraded":"2024-02-21T23:53:24.493527+0000","last_fullsized":"2024-02-21T23:53:24.493527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'5","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:01.175408+0000","last_change":"2024-02-21T23:25:16.129446+0000","last_active":"2024-02-21T23:53:01.175408+0000","last_peered":"2024-02-21T23:53:01.175408+0000","last_clean":"2024-02-21T23:53:01.175408+0000","last_became_active":"2024-02-21T23:25:16.129164+0000","last_became_peered":"2024-02-21T23:25:16.129164+0000","last_unstale":"2024-02-21T23:53:01.175408+0000","last_undegraded":"2024-02-21T23:53:01.175408+0000","last_fullsized":"2024-02-21T23:53:01.175408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'139","reported_seq":162,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.403580+0000","last_change":"2024-02-21T23:25:57.600307+0000","last_active":"2024-02-21T23:52:56.403580+0000","last_peered":"2024-02-21T23:52:56.403580+0000","last_clean":"2024-02-21T23:52:56.403580+0000","last_became_active":"2024-02-21T23:25:57.599867+0000","last_became_peered":"2024-02-21T23:25:57.599867+0000","last_unstale":"2024-02-21T23:52:56.403580+0000","last_undegraded":"2024-02-21T23:52:56.403580+0000","last_fullsized":"2024-02-21T23:52:56.403580+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:39.763036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":5393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,5],"acting":[2,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"26'153","reported_seq":185,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.092755+0000","last_change":"2024-02-21T23:25:22.248119+0000","last_active":"2024-02-21T23:52:55.092755+0000","last_peered":"2024-02-21T23:52:55.092755+0000","last_clean":"2024-02-21T23:52:55.092755+0000","last_became_active":"2024-02-21T23:25:22.247811+0000","last_became_peered":"2024-02-21T23:25:22.247811+0000","last_unstale":"2024-02-21T23:52:55.092755+0000","last_undegraded":"2024-02-21T23:52:55.092755+0000","last_fullsized":"2024-02-21T23:52:55.092755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":11393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'41","reported_seq":76,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.466267+0000","last_change":"2024-02-21T23:25:17.189176+0000","last_active":"2024-02-21T23:53:24.466267+0000","last_peered":"2024-02-21T23:53:24.466267+0000","last_clean":"2024-02-21T23:53:24.466267+0000","last_became_active":"2024-02-21T23:25:17.188885+0000","last_became_peered":"2024-02-21T23:25:17.188885+0000","last_unstale":"2024-02-21T23:53:24.466267+0000","last_undegraded":"2024-02-21T23:53:24.466267+0000","last_fullsized":"2024-02-21T23:53:24.466267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'4","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:04.260921+0000","last_change":"2024-02-21T23:25:16.132036+0000","last_active":"2024-02-21T23:53:04.260921+0000","last_peered":"2024-02-21T23:53:04.260921+0000","last_clean":"2024-02-21T23:53:04.260921+0000","last_became_active":"2024-02-21T23:25:16.131756+0000","last_became_peered":"2024-02-21T23:25:16.131756+0000","last_unstale":"2024-02-21T23:53:04.260921+0000","last_undegraded":"2024-02-21T23:53:04.260921+0000","last_fullsized":"2024-02-21T23:53:04.260921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"26'157","reported_seq":188,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.421511+0000","last_change":"2024-02-21T23:25:21.686133+0000","last_active":"2024-02-21T23:52:56.421511+0000","last_peered":"2024-02-21T23:52:56.421511+0000","last_clean":"2024-02-21T23:52:56.421511+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:52:56.421511+0000","last_undegraded":"2024-02-21T23:52:56.421511+0000","last_fullsized":"2024-02-21T23:52:56.421511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":15074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'44","reported_seq":79,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471213+0000","last_change":"2024-02-21T23:25:17.135158+0000","last_active":"2024-02-21T23:53:24.471213+0000","last_peered":"2024-02-21T23:53:24.471213+0000","last_clean":"2024-02-21T23:53:24.471213+0000","last_became_active":"2024-02-21T23:25:17.134888+0000","last_became_peered":"2024-02-21T23:25:17.134888+0000","last_unstale":"2024-02-21T23:53:24.471213+0000","last_undegraded":"2024-02-21T23:53:24.471213+0000","last_fullsized":"2024-02-21T23:53:24.471213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:32:49.378077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'5","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:10.598178+0000","last_change":"2024-02-21T23:25:16.116378+0000","last_active":"2024-02-21T23:53:10.598178+0000","last_peered":"2024-02-21T23:53:10.598178+0000","last_clean":"2024-02-21T23:53:10.598178+0000","last_became_active":"2024-02-21T23:25:16.116201+0000","last_became_peered":"2024-02-21T23:25:16.116201+0000","last_unstale":"2024-02-21T23:53:10.598178+0000","last_undegraded":"2024-02-21T23:53:10.598178+0000","last_fullsized":"2024-02-21T23:53:10.598178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:28:45.946585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'718","reported_seq":755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.456812+0000","last_change":"2024-02-21T23:25:16.132314+0000","last_active":"2024-02-21T23:52:48.456812+0000","last_peered":"2024-02-21T23:52:48.456812+0000","last_clean":"2024-02-21T23:52:48.456812+0000","last_became_active":"2024-02-21T23:25:16.132062+0000","last_became_peered":"2024-02-21T23:25:16.132062+0000","last_unstale":"2024-02-21T23:52:48.456812+0000","last_undegraded":"2024-02-21T23:52:48.456812+0000","last_fullsized":"2024-02-21T23:52:48.456812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":716,"num_write_kb":4453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'39","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607247+0000","last_change":"2024-02-21T23:25:17.161451+0000","last_active":"2024-02-21T23:53:24.607247+0000","last_peered":"2024-02-21T23:53:24.607247+0000","last_clean":"2024-02-21T23:53:24.607247+0000","last_became_active":"2024-02-21T23:25:17.161176+0000","last_became_peered":"2024-02-21T23:25:17.161176+0000","last_unstale":"2024-02-21T23:53:24.607247+0000","last_undegraded":"2024-02-21T23:53:24.607247+0000","last_fullsized":"2024-02-21T23:53:24.607247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'95","reported_seq":124,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:53.621659+0000","last_change":"2024-02-21T23:25:21.685724+0000","last_active":"2024-02-21T23:52:53.621659+0000","last_peered":"2024-02-21T23:52:53.621659+0000","last_clean":"2024-02-21T23:52:53.621659+0000","last_became_active":"2024-02-21T23:25:21.685375+0000","last_became_peered":"2024-02-21T23:25:21.685375+0000","last_unstale":"2024-02-21T23:52:53.621659+0000","last_undegraded":"2024-02-21T23:52:53.621659+0000","last_fullsized":"2024-02-21T23:52:53.621659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:56:28.795401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":7997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"26'688","reported_seq":725,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:06.343128+0000","last_change":"2024-02-21T23:25:16.134127+0000","last_active":"2024-02-21T23:53:06.343128+0000","last_peered":"2024-02-21T23:53:06.343128+0000","last_clean":"2024-02-21T23:53:06.343128+0000","last_became_active":"2024-02-21T23:25:16.133839+0000","last_became_peered":"2024-02-21T23:25:16.133839+0000","last_unstale":"2024-02-21T23:53:06.343128+0000","last_undegraded":"2024-02-21T23:53:06.343128+0000","last_fullsized":"2024-02-21T23:53:06.343128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:31:45.308649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'33","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.483697+0000","last_change":"2024-02-21T23:25:17.190455+0000","last_active":"2024-02-21T23:53:24.483697+0000","last_peered":"2024-02-21T23:53:24.483697+0000","last_clean":"2024-02-21T23:53:24.483697+0000","last_became_active":"2024-02-21T23:25:17.190194+0000","last_became_peered":"2024-02-21T23:25:17.190194+0000","last_unstale":"2024-02-21T23:53:24.483697+0000","last_undegraded":"2024-02-21T23:53:24.483697+0000","last_fullsized":"2024-02-21T23:53:24.483697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:03.451409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'159","reported_seq":195,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.988198+0000","last_change":"2024-02-21T23:25:57.526175+0000","last_active":"2024-02-21T23:52:52.988198+0000","last_peered":"2024-02-21T23:52:52.988198+0000","last_clean":"2024-02-21T23:52:52.988198+0000","last_became_active":"2024-02-21T23:25:57.525755+0000","last_became_peered":"2024-02-21T23:25:57.525755+0000","last_unstale":"2024-02-21T23:52:52.988198+0000","last_undegraded":"2024-02-21T23:52:52.988198+0000","last_fullsized":"2024-02-21T23:52:52.988198+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:18:17.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4,2],"acting":[3,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'48","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626470+0000","last_change":"2024-02-21T23:25:17.138039+0000","last_active":"2024-02-21T23:53:24.626470+0000","last_peered":"2024-02-21T23:53:24.626470+0000","last_clean":"2024-02-21T23:53:24.626470+0000","last_became_active":"2024-02-21T23:25:17.137717+0000","last_became_peered":"2024-02-21T23:25:17.137717+0000","last_unstale":"2024-02-21T23:53:24.626470+0000","last_undegraded":"2024-02-21T23:53:24.626470+0000","last_fullsized":"2024-02-21T23:53:24.626470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"26'48","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.529559+0000","last_change":"2024-02-21T23:25:17.194727+0000","last_active":"2024-02-21T23:53:24.529559+0000","last_peered":"2024-02-21T23:53:24.529559+0000","last_clean":"2024-02-21T23:53:24.529559+0000","last_became_active":"2024-02-21T23:25:17.194533+0000","last_became_peered":"2024-02-21T23:25:17.194533+0000","last_unstale":"2024-02-21T23:53:24.529559+0000","last_undegraded":"2024-02-21T23:53:24.529559+0000","last_fullsized":"2024-02-21T23:53:24.529559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:42.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"26'50","reported_seq":87,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493574+0000","last_change":"2024-02-21T23:25:17.191051+0000","last_active":"2024-02-21T23:53:24.493574+0000","last_peered":"2024-02-21T23:53:24.493574+0000","last_clean":"2024-02-21T23:53:24.493574+0000","last_became_active":"2024-02-21T23:25:17.190600+0000","last_became_peered":"2024-02-21T23:25:17.190600+0000","last_unstale":"2024-02-21T23:53:24.493574+0000","last_undegraded":"2024-02-21T23:53:24.493574+0000","last_fullsized":"2024-02-21T23:53:24.493574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.599153+0000","last_change":"2024-02-21T23:25:17.135311+0000","last_active":"2024-02-21T23:53:24.599153+0000","last_peered":"2024-02-21T23:53:24.599153+0000","last_clean":"2024-02-21T23:53:24.599153+0000","last_became_active":"2024-02-21T23:25:17.135011+0000","last_became_peered":"2024-02-21T23:25:17.135011+0000","last_unstale":"2024-02-21T23:53:24.599153+0000","last_undegraded":"2024-02-21T23:53:24.599153+0000","last_fullsized":"2024-02-21T23:53:24.599153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.472250+0000","last_change":"2024-02-21T23:25:17.161696+0000","last_active":"2024-02-21T23:53:24.472250+0000","last_peered":"2024-02-21T23:53:24.472250+0000","last_clean":"2024-02-21T23:53:24.472250+0000","last_became_active":"2024-02-21T23:25:17.161401+0000","last_became_peered":"2024-02-21T23:25:17.161401+0000","last_unstale":"2024-02-21T23:53:24.472250+0000","last_undegraded":"2024-02-21T23:53:24.472250+0000","last_fullsized":"2024-02-21T23:53:24.472250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"26'34","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.463178+0000","last_change":"2024-02-21T23:25:17.189731+0000","last_active":"2024-02-21T23:53:24.463178+0000","last_peered":"2024-02-21T23:53:24.463178+0000","last_clean":"2024-02-21T23:53:24.463178+0000","last_became_active":"2024-02-21T23:25:17.189484+0000","last_became_peered":"2024-02-21T23:25:17.189484+0000","last_unstale":"2024-02-21T23:53:24.463178+0000","last_undegraded":"2024-02-21T23:53:24.463178+0000","last_fullsized":"2024-02-21T23:53:24.463178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"26'45","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.628365+0000","last_change":"2024-02-21T23:25:17.184125+0000","last_active":"2024-02-21T23:53:24.628365+0000","last_peered":"2024-02-21T23:53:24.628365+0000","last_clean":"2024-02-21T23:53:24.628365+0000","last_became_active":"2024-02-21T23:25:17.183865+0000","last_became_peered":"2024-02-21T23:25:17.183865+0000","last_unstale":"2024-02-21T23:53:24.628365+0000","last_undegraded":"2024-02-21T23:53:24.628365+0000","last_fullsized":"2024-02-21T23:53:24.628365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'37","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.468066+0000","last_change":"2024-02-21T23:25:17.161482+0000","last_active":"2024-02-21T23:53:24.468066+0000","last_peered":"2024-02-21T23:53:24.468066+0000","last_clean":"2024-02-21T23:53:24.468066+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:24.468066+0000","last_undegraded":"2024-02-21T23:53:24.468066+0000","last_fullsized":"2024-02-21T23:53:24.468066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'32","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494217+0000","last_change":"2024-02-21T23:25:17.185691+0000","last_active":"2024-02-21T23:53:24.494217+0000","last_peered":"2024-02-21T23:53:24.494217+0000","last_clean":"2024-02-21T23:53:24.494217+0000","last_became_active":"2024-02-21T23:25:17.185420+0000","last_became_peered":"2024-02-21T23:25:17.185420+0000","last_unstale":"2024-02-21T23:53:24.494217+0000","last_undegraded":"2024-02-21T23:53:24.494217+0000","last_fullsized":"2024-02-21T23:53:24.494217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"26'46","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471138+0000","last_change":"2024-02-21T23:25:17.161321+0000","last_active":"2024-02-21T23:53:24.471138+0000","last_peered":"2024-02-21T23:53:24.471138+0000","last_clean":"2024-02-21T23:53:24.471138+0000","last_became_active":"2024-02-21T23:25:17.161010+0000","last_became_peered":"2024-02-21T23:25:17.161010+0000","last_unstale":"2024-02-21T23:53:24.471138+0000","last_undegraded":"2024-02-21T23:53:24.471138+0000","last_fullsized":"2024-02-21T23:53:24.471138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"26'39","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494124+0000","last_change":"2024-02-21T23:25:17.191072+0000","last_active":"2024-02-21T23:53:24.494124+0000","last_peered":"2024-02-21T23:53:24.494124+0000","last_clean":"2024-02-21T23:53:24.494124+0000","last_became_active":"2024-02-21T23:25:17.187741+0000","last_became_peered":"2024-02-21T23:25:17.187741+0000","last_unstale":"2024-02-21T23:53:24.494124+0000","last_undegraded":"2024-02-21T23:53:24.494124+0000","last_fullsized":"2024-02-21T23:53:24.494124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"26'53","reported_seq":89,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607148+0000","last_change":"2024-02-21T23:25:17.186616+0000","last_active":"2024-02-21T23:53:24.607148+0000","last_peered":"2024-02-21T23:53:24.607148+0000","last_clean":"2024-02-21T23:53:24.607148+0000","last_became_active":"2024-02-21T23:25:17.186108+0000","last_became_peered":"2024-02-21T23:25:17.186108+0000","last_unstale":"2024-02-21T23:53:24.607148+0000","last_undegraded":"2024-02-21T23:53:24.607148+0000","last_fullsized":"2024-02-21T23:53:24.607148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"26'43","reported_seq":79,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626297+0000","last_change":"2024-02-21T23:25:17.161428+0000","last_active":"2024-02-21T23:53:24.626297+0000","last_peered":"2024-02-21T23:53:24.626297+0000","last_clean":"2024-02-21T23:53:24.626297+0000","last_became_active":"2024-02-21T23:25:17.161140+0000","last_became_peered":"2024-02-21T23:25:17.161140+0000","last_unstale":"2024-02-21T23:53:24.626297+0000","last_undegraded":"2024-02-21T23:53:24.626297+0000","last_fullsized":"2024-02-21T23:53:24.626297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604720+0000","last_change":"2024-02-21T23:25:17.190666+0000","last_active":"2024-02-21T23:53:24.604720+0000","last_peered":"2024-02-21T23:53:24.604720+0000","last_clean":"2024-02-21T23:53:24.604720+0000","last_became_active":"2024-02-21T23:25:17.190420+0000","last_became_peered":"2024-02-21T23:25:17.190420+0000","last_unstale":"2024-02-21T23:53:24.604720+0000","last_undegraded":"2024-02-21T23:53:24.604720+0000","last_fullsized":"2024-02-21T23:53:24.604720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"26'48","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.531320+0000","last_change":"2024-02-21T23:25:17.189583+0000","last_active":"2024-02-21T23:53:24.531320+0000","last_peered":"2024-02-21T23:53:24.531320+0000","last_clean":"2024-02-21T23:53:24.531320+0000","last_became_active":"2024-02-21T23:25:17.189302+0000","last_became_peered":"2024-02-21T23:25:17.189302+0000","last_unstale":"2024-02-21T23:53:24.531320+0000","last_undegraded":"2024-02-21T23:53:24.531320+0000","last_fullsized":"2024-02-21T23:53:24.531320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"26'41","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493492+0000","last_change":"2024-02-21T23:25:17.188113+0000","last_active":"2024-02-21T23:53:24.493492+0000","last_peered":"2024-02-21T23:53:24.493492+0000","last_clean":"2024-02-21T23:53:24.493492+0000","last_became_active":"2024-02-21T23:25:17.187811+0000","last_became_peered":"2024-02-21T23:25:17.187811+0000","last_unstale":"2024-02-21T23:53:24.493492+0000","last_undegraded":"2024-02-21T23:53:24.493492+0000","last_fullsized":"2024-02-21T23:53:24.493492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"26'44","reported_seq":65,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.530148+0000","last_change":"2024-02-21T23:25:57.525755+0000","last_active":"2024-02-21T23:53:24.530148+0000","last_peered":"2024-02-21T23:53:24.530148+0000","last_clean":"2024-02-21T23:53:24.530148+0000","last_became_active":"2024-02-21T23:25:57.525443+0000","last_became_peered":"2024-02-21T23:25:57.525443+0000","last_unstale":"2024-02-21T23:53:24.530148+0000","last_undegraded":"2024-02-21T23:53:24.530148+0000","last_fullsized":"2024-02-21T23:53:24.530148+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.457278+0000","last_change":"2024-02-21T23:25:17.186378+0000","last_active":"2024-02-21T23:53:24.457278+0000","last_peered":"2024-02-21T23:53:24.457278+0000","last_clean":"2024-02-21T23:53:24.457278+0000","last_became_active":"2024-02-21T23:25:17.186086+0000","last_became_peered":"2024-02-21T23:25:17.186086+0000","last_unstale":"2024-02-21T23:53:24.457278+0000","last_undegraded":"2024-02-21T23:53:24.457278+0000","last_fullsized":"2024-02-21T23:53:24.457278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"26'41","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.602175+0000","last_change":"2024-02-21T23:25:17.190934+0000","last_active":"2024-02-21T23:53:24.602175+0000","last_peered":"2024-02-21T23:53:24.602175+0000","last_clean":"2024-02-21T23:53:24.602175+0000","last_became_active":"2024-02-21T23:25:17.190680+0000","last_became_peered":"2024-02-21T23:25:17.190680+0000","last_unstale":"2024-02-21T23:53:24.602175+0000","last_undegraded":"2024-02-21T23:53:24.602175+0000","last_fullsized":"2024-02-21T23:53:24.602175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:22:04.238374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"26'38","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.473047+0000","last_change":"2024-02-21T23:25:17.135140+0000","last_active":"2024-02-21T23:53:24.473047+0000","last_peered":"2024-02-21T23:53:24.473047+0000","last_clean":"2024-02-21T23:53:24.473047+0000","last_became_active":"2024-02-21T23:25:17.134866+0000","last_became_peered":"2024-02-21T23:25:17.134866+0000","last_unstale":"2024-02-21T23:53:24.473047+0000","last_undegraded":"2024-02-21T23:53:24.473047+0000","last_fullsized":"2024-02-21T23:53:24.473047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'27","reported_seq":62,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.488701+0000","last_change":"2024-02-21T23:25:17.162204+0000","last_active":"2024-02-21T23:53:24.488701+0000","last_peered":"2024-02-21T23:53:24.488701+0000","last_clean":"2024-02-21T23:53:24.488701+0000","last_became_active":"2024-02-21T23:25:17.161807+0000","last_became_peered":"2024-02-21T23:25:17.161807+0000","last_unstale":"2024-02-21T23:53:24.488701+0000","last_undegraded":"2024-02-21T23:53:24.488701+0000","last_fullsized":"2024-02-21T23:53:24.488701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:26.819873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"26'36","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626279+0000","last_change":"2024-02-21T23:25:17.188932+0000","last_active":"2024-02-21T23:53:24.626279+0000","last_peered":"2024-02-21T23:53:24.626279+0000","last_clean":"2024-02-21T23:53:24.626279+0000","last_became_active":"2024-02-21T23:25:17.188646+0000","last_became_peered":"2024-02-21T23:25:17.188646+0000","last_unstale":"2024-02-21T23:53:24.626279+0000","last_undegraded":"2024-02-21T23:53:24.626279+0000","last_fullsized":"2024-02-21T23:53:24.626279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"26'51","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.628333+0000","last_change":"2024-02-21T23:25:17.188333+0000","last_active":"2024-02-21T23:53:24.628333+0000","last_peered":"2024-02-21T23:53:24.628333+0000","last_clean":"2024-02-21T23:53:24.628333+0000","last_became_active":"2024-02-21T23:25:17.188066+0000","last_became_peered":"2024-02-21T23:25:17.188066+0000","last_unstale":"2024-02-21T23:53:24.628333+0000","last_undegraded":"2024-02-21T23:53:24.628333+0000","last_fullsized":"2024-02-21T23:53:24.628333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"26'39","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605416+0000","last_change":"2024-02-21T23:25:17.187549+0000","last_active":"2024-02-21T23:53:24.605416+0000","last_peered":"2024-02-21T23:53:24.605416+0000","last_clean":"2024-02-21T23:53:24.605416+0000","last_became_active":"2024-02-21T23:25:17.187297+0000","last_became_peered":"2024-02-21T23:25:17.187297+0000","last_unstale":"2024-02-21T23:53:24.605416+0000","last_undegraded":"2024-02-21T23:53:24.605416+0000","last_fullsized":"2024-02-21T23:53:24.605416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"26'30","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.535065+0000","last_change":"2024-02-21T23:25:57.526373+0000","last_active":"2024-02-21T23:53:24.535065+0000","last_peered":"2024-02-21T23:53:24.535065+0000","last_clean":"2024-02-21T23:53:24.535065+0000","last_became_active":"2024-02-21T23:25:57.526068+0000","last_became_peered":"2024-02-21T23:25:57.526068+0000","last_unstale":"2024-02-21T23:53:24.535065+0000","last_undegraded":"2024-02-21T23:53:24.535065+0000","last_fullsized":"2024-02-21T23:53:24.535065+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"26'37","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.597841+0000","last_change":"2024-02-21T23:25:17.183264+0000","last_active":"2024-02-21T23:53:24.597841+0000","last_peered":"2024-02-21T23:53:24.597841+0000","last_clean":"2024-02-21T23:53:24.597841+0000","last_became_active":"2024-02-21T23:25:17.182924+0000","last_became_peered":"2024-02-21T23:25:17.182924+0000","last_unstale":"2024-02-21T23:53:24.597841+0000","last_undegraded":"2024-02-21T23:53:24.597841+0000","last_fullsized":"2024-02-21T23:53:24.597841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"26'38","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494000+0000","last_change":"2024-02-21T23:25:17.190841+0000","last_active":"2024-02-21T23:53:24.494000+0000","last_peered":"2024-02-21T23:53:24.494000+0000","last_clean":"2024-02-21T23:53:24.494000+0000","last_became_active":"2024-02-21T23:25:17.190276+0000","last_became_peered":"2024-02-21T23:25:17.190276+0000","last_unstale":"2024-02-21T23:53:24.494000+0000","last_undegraded":"2024-02-21T23:53:24.494000+0000","last_fullsized":"2024-02-21T23:53:24.494000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'36","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.601860+0000","last_change":"2024-02-21T23:25:17.189872+0000","last_active":"2024-02-21T23:53:24.601860+0000","last_peered":"2024-02-21T23:53:24.601860+0000","last_clean":"2024-02-21T23:53:24.601860+0000","last_became_active":"2024-02-21T23:25:17.189614+0000","last_became_peered":"2024-02-21T23:25:17.189614+0000","last_unstale":"2024-02-21T23:53:24.601860+0000","last_undegraded":"2024-02-21T23:53:24.601860+0000","last_fullsized":"2024-02-21T23:53:24.601860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"26'25","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493930+0000","last_change":"2024-02-21T23:25:57.600458+0000","last_active":"2024-02-21T23:53:24.493930+0000","last_peered":"2024-02-21T23:53:24.493930+0000","last_clean":"2024-02-21T23:53:24.493930+0000","last_became_active":"2024-02-21T23:25:57.600094+0000","last_became_peered":"2024-02-21T23:25:57.600094+0000","last_unstale":"2024-02-21T23:53:24.493930+0000","last_undegraded":"2024-02-21T23:53:24.493930+0000","last_fullsized":"2024-02-21T23:53:24.493930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:20:16.177299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"26'44","reported_seq":81,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.602153+0000","last_change":"2024-02-21T23:25:17.191159+0000","last_active":"2024-02-21T23:53:24.602153+0000","last_peered":"2024-02-21T23:53:24.602153+0000","last_clean":"2024-02-21T23:53:24.602153+0000","last_became_active":"2024-02-21T23:25:17.190576+0000","last_became_peered":"2024-02-21T23:25:17.190576+0000","last_unstale":"2024-02-21T23:53:24.602153+0000","last_undegraded":"2024-02-21T23:53:24.602153+0000","last_fullsized":"2024-02-21T23:53:24.602153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:13:18.524027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"26'34","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.452090+0000","last_change":"2024-02-21T23:25:17.189287+0000","last_active":"2024-02-21T23:53:24.452090+0000","last_peered":"2024-02-21T23:53:24.452090+0000","last_clean":"2024-02-21T23:53:24.452090+0000","last_became_active":"2024-02-21T23:25:17.189029+0000","last_became_peered":"2024-02-21T23:25:17.189029+0000","last_unstale":"2024-02-21T23:53:24.452090+0000","last_undegraded":"2024-02-21T23:53:24.452090+0000","last_fullsized":"2024-02-21T23:53:24.452090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605166+0000","last_change":"2024-02-21T23:25:17.183737+0000","last_active":"2024-02-21T23:53:24.605166+0000","last_peered":"2024-02-21T23:53:24.605166+0000","last_clean":"2024-02-21T23:53:24.605166+0000","last_became_active":"2024-02-21T23:25:17.183410+0000","last_became_peered":"2024-02-21T23:25:17.183410+0000","last_unstale":"2024-02-21T23:53:24.605166+0000","last_undegraded":"2024-02-21T23:53:24.605166+0000","last_fullsized":"2024-02-21T23:53:24.605166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4087,"num_write_kb":323834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4616,"ondisk_log_size":4616,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2419,"ondisk_log_size":2419,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":40532172,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":11557,"num_write_kb":45747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55631872,"data_stored":81064344,"data_compressed":7543912,"data_compressed_allocated":11722752,"data_compressed_original":37216256,"omap_allocated":0,"internal_metadata":0},"log_size":9530,"ondisk_log_size":9530,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575193,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78552,"kb_used_data":16848,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94293288,"statfs":{"total":96636764160,"available":96556326912,"internally_reserved":0,"allocated":17252352,"data_stored":25871404,"data_compressed":2449552,"data_compressed_allocated":3768320,"data_compressed_original":12558336,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575193,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61012,"kb_used_data":11788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310828,"statfs":{"total":96636764160,"available":96574287872,"internally_reserved":0,"allocated":12070912,"data_stored":16892518,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575193,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":114720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575193,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60916,"kb_used_data":7532,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310924,"statfs":{"total":96636764160,"available":96574386176,"internally_reserved":0,"allocated":7712768,"data_stored":11064847,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575193,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58952,"kb_used_data":9728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312888,"statfs":{"total":96636764160,"available":96576397312,"internally_reserved":0,"allocated":9961472,"data_stored":12918697,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575193,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":495616,"data_stored":335113,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575193,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47824,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324016,"statfs":{"total":96636764160,"available":96587792384,"internally_reserved":0,"allocated":2826240,"data_stored":4309094,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575193,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60600,"kb_used_data":7216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311240,"statfs":{"total":96636764160,"available":96574709760,"internally_reserved":0,"allocated":7389184,"data_stored":11655727,"data_compressed":1158786,"data_compressed_allocated":1720320,"data_compressed_original":6168576,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7045120,"data_stored":10950729,"data_compressed":1126387,"data_compressed_allocated":1675264,"data_compressed_original":5595136,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2547712,"data_stored":4194464,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":220483,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9682944,"data_stored":12804067,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7434240,"data_stored":10950217,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11792384,"data_stored":16777888,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16908288,"data_stored":25166406,"data_compressed":2417153,"data_compressed_allocated":3723264,"data_compressed_original":11984896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-21T23:53:32.449 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:53:32.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.591+0000 7feb34155640 1 Processor -- start 2024-02-21T23:53:32.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.591+0000 7feb34155640 1 -- start start 2024-02-21T23:53:32.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.592+0000 7feb34155640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feb2c0c41a0 0x7feb2c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:32.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.592+0000 7feb34155640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c156e90 con 0x7feb2c150d30 2024-02-21T23:53:32.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.592+0000 7feb34155640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c157150 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.592+0000 7feb34155640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c157430 con 0x7feb2c0c41a0 2024-02-21T23:53:32.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.592+0000 7feb316f0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feb2c0c41a0 0x7feb2c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:32.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.592+0000 7feb31ef1640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7feb2c0c3dc0 0x7feb2c070410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57462/0 (socket says 172.21.15.133:57462) 2024-02-21T23:53:32.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.592+0000 7feb31ef1640 1 -- 172.21.15.133:0/3017978161 learned_addr learned my addr 172.21.15.133:0/3017978161 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:32.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.595+0000 7feb30eef640 1 -- 172.21.15.133:0/3017978161 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985726086 0 0) 0x7feb2c157150 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.595+0000 7feb30eef640 1 -- 172.21.15.133:0/3017978161 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb18003520 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.597+0000 7feb316f0640 1 -- 172.21.15.133:0/3017978161 >> v1:172.21.15.142:6789/0 conn(0x7feb2c0c3dc0 legacy=0x7feb2c070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.597+0000 7feb316f0640 1 -- 172.21.15.133:0/3017978161 >> v1:172.21.15.133:6789/0 conn(0x7feb2c150d30 legacy=0x7feb2c15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.597+0000 7feb316f0640 1 -- 172.21.15.133:0/3017978161 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c157710 con 0x7feb2c0c41a0 2024-02-21T23:53:32.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.600+0000 7feb316f0640 1 --2- 172.21.15.133:0/3017978161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb2c0c41a0 0x7feb2c0cd010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0097a0 tx=0x7feb1c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2721db69a3dcf3 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:32.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.605+0000 7feb30eef640 1 -- 172.21.15.133:0/3017978161 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c034070 con 0x7feb2c0c41a0 2024-02-21T23:53:32.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.605+0000 7feb30eef640 1 -- 172.21.15.133:0/3017978161 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c030610 con 0x7feb2c0c41a0 2024-02-21T23:53:32.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.605+0000 7feb30eef640 1 -- 172.21.15.133:0/3017978161 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0309e0 con 0x7feb2c0c41a0 2024-02-21T23:53:32.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.605+0000 7feb34155640 1 -- 172.21.15.133:0/3017978161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb2c0c41a0 msgr2=0x7feb2c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:32.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.605+0000 7feb34155640 1 --2- 172.21.15.133:0/3017978161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb2c0c41a0 0x7feb2c0cd010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0097a0 tx=0x7feb1c02b960 comp rx=0 tx=0).stop 2024-02-21T23:53:32.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.606+0000 7feb34155640 1 -- 172.21.15.133:0/3017978161 shutdown_connections 2024-02-21T23:53:32.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.606+0000 7feb34155640 1 --2- 172.21.15.133:0/3017978161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb2c0c41a0 0x7feb2c0cd010 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:32.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.606+0000 7feb34155640 1 -- 172.21.15.133:0/3017978161 >> 172.21.15.133:0/3017978161 conn(0x7feb2c061b50 msgr2=0x7feb2c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:32.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.606+0000 7feb34155640 1 -- 172.21.15.133:0/3017978161 shutdown_connections 2024-02-21T23:53:32.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.606+0000 7feb34155640 1 -- 172.21.15.133:0/3017978161 wait complete. 2024-02-21T23:53:32.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.607+0000 7feb34155640 1 Processor -- start 2024-02-21T23:53:32.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.607+0000 7feb34155640 1 -- start start 2024-02-21T23:53:32.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb34155640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c0c3dc0 0x7feb2c159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:32.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb34155640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb2c0c41a0 0x7feb2c0c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:32.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb34155640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c150d30 0x7feb2c14f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:32.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb34155640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c070190 con 0x7feb2c0c41a0 2024-02-21T23:53:32.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb34155640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c157cd0 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb34155640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c15ad10 con 0x7feb2c150d30 2024-02-21T23:53:32.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb316f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb2c0c41a0 0x7feb2c0c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:32.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb31ef1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c0c3dc0 0x7feb2c159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:32.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb326f2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c150d30 0x7feb2c14f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:32.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb31ef1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c0c3dc0 0x7feb2c159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58818/0 (socket says 172.21.15.133:58818) 2024-02-21T23:53:32.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.608+0000 7feb31ef1640 1 -- 172.21.15.133:0/3601025577 learned_addr learned my addr 172.21.15.133:0/3601025577 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:32.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.612+0000 7feb31ef1640 1 -- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c150d30 msgr2=0x7feb2c14f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:32.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.612+0000 7feb31ef1640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c150d30 0x7feb2c14f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:32.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.612+0000 7feb31ef1640 1 -- 172.21.15.133:0/3601025577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb2c0c41a0 msgr2=0x7feb2c0c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:32.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.612+0000 7feb31ef1640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb2c0c41a0 0x7feb2c0c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:32.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.612+0000 7feb31ef1640 1 -- 172.21.15.133:0/3601025577 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c1e2b10 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.614+0000 7feb31ef1640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c0c3dc0 0x7feb2c159770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c560 tx=0x7feb2800ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.618+0000 7feb227fc640 1 -- 172.21.15.133:0/3601025577 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28017070 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.618+0000 7feb227fc640 1 -- 172.21.15.133:0/3601025577 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb28005ce0 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.618+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c0cb990 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.618+0000 7feb227fc640 1 -- 172.21.15.133:0/3601025577 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280072b0 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.618+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c0cbf10 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.619+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c06a130 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.886+0000 7feb227fc640 1 -- 172.21.15.133:0/3601025577 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb2801a070 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.886+0000 7feb227fc640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feb1006b520 0x7feb1006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.887+0000 7feb316f0640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feb1006b520 0x7feb1006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.887+0000 7feb316f0640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feb1006b520 0x7feb1006d9e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb1c036000 tx=0x7feb1c03b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:32.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.890+0000 7feb227fc640 1 -- 172.21.15.133:0/3601025577 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7feb28056340 con 0x7feb2c0c3dc0 2024-02-21T23:53:32.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:32.901+0000 7feb227fc640 1 -- 172.21.15.133:0/3601025577 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7feb28021020 con 0x7feb2c0c3dc0 2024-02-21T23:53:33.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.100+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb2c0c7ce0 con 0x7feb2c0c3dc0 2024-02-21T23:53:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.140+0000 7feb227fc640 1 -- 172.21.15.133:0/3601025577 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19091 (secure 0 0 0) 0x7feb280582a0 con 0x7feb2c0c3dc0 2024-02-21T23:53:33.136 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:53:33.136 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:57.029812+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":0}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:53:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feb1006b520 msgr2=0x7feb1006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feb1006b520 0x7feb1006d9e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb1c036000 tx=0x7feb1c03b040 comp rx=0 tx=0).stop 2024-02-21T23:53:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c0c3dc0 msgr2=0x7feb2c159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c0c3dc0 0x7feb2c159770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c560 tx=0x7feb2800ca20 comp rx=0 tx=0).stop 2024-02-21T23:53:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 shutdown_connections 2024-02-21T23:53:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feb1006b520 0x7feb1006d9e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb2c150d30 0x7feb2c14f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:33.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb2c0c41a0 0x7feb2c0c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:33.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 --2- 172.21.15.133:0/3601025577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2c0c3dc0 0x7feb2c159770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:33.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.142+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 >> 172.21.15.133:0/3601025577 conn(0x7feb2c061b50 msgr2=0x7feb2c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:33.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.143+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 shutdown_connections 2024-02-21T23:53:33.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:33.143+0000 7feb34155640 1 -- 172.21.15.133:0/3601025577 wait complete. 2024-02-21T23:53:33.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:53:33.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:53:34.151 INFO:tasks.ceph:Scrubbing osd.0 2024-02-21T23:53:34.151 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-02-21T23:53:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.264+0000 7f6058d7b640 1 Processor -- start 2024-02-21T23:53:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.264+0000 7f6058d7b640 1 -- start start 2024-02-21T23:53:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.264+0000 7f6058d7b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6054154600 0x7f60540cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.264+0000 7f6058d7b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60540c6f50 0x7f60540ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.264+0000 7f6058d7b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605415ab40 con 0x7f60540cab80 2024-02-21T23:53:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.264+0000 7f6058d7b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60540c74a0 con 0x7f6054154600 2024-02-21T23:53:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.264+0000 7f6058d7b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60540c7780 con 0x7f60540c6f50 2024-02-21T23:53:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.265+0000 7f605259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6054154600 0x7f60540cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.265+0000 7f604bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60540c6f50 0x7f60540ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.265+0000 7f604bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60540c6f50 0x7f60540ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:47862/0 (socket says 172.21.15.133:47862) 2024-02-21T23:53:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.265+0000 7f604bfff640 1 -- 172.21.15.133:0/4151844084 learned_addr learned my addr 172.21.15.133:0/4151844084 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.268+0000 7f604bfff640 1 -- 172.21.15.133:0/4151844084 >> v2:172.21.15.133:3300/0 conn(0x7f6054154600 msgr2=0x7f60540cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.268+0000 7f604bfff640 1 --2- 172.21.15.133:0/4151844084 >> v2:172.21.15.133:3300/0 conn(0x7f6054154600 0x7f60540cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.268+0000 7f604bfff640 1 -- 172.21.15.133:0/4151844084 >> v1:172.21.15.133:6789/0 conn(0x7f60540cab80 legacy=0x7f60540c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.268+0000 7f604bfff640 1 -- 172.21.15.133:0/4151844084 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605415ae00 con 0x7f60540c6f50 2024-02-21T23:53:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.270+0000 7f604bfff640 1 --2- 172.21.15.133:0/4151844084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60540c6f50 0x7f60540ca640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f603c009870 tx=0x7f603c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e746df8d1ee58520 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:34.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.273+0000 7f6051d9b640 1 -- 172.21.15.133:0/4151844084 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c034070 con 0x7f60540c6f50 2024-02-21T23:53:34.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.273+0000 7f6051d9b640 1 -- 172.21.15.133:0/4151844084 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c030840 con 0x7f60540c6f50 2024-02-21T23:53:34.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.273+0000 7f6051d9b640 1 -- 172.21.15.133:0/4151844084 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c030c10 con 0x7f60540c6f50 2024-02-21T23:53:34.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.274+0000 7f6058d7b640 1 -- 172.21.15.133:0/4151844084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60540c6f50 msgr2=0x7f60540ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.274+0000 7f6058d7b640 1 --2- 172.21.15.133:0/4151844084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60540c6f50 0x7f60540ca640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f603c009870 tx=0x7f603c02b980 comp rx=0 tx=0).stop 2024-02-21T23:53:34.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.274+0000 7f6058d7b640 1 -- 172.21.15.133:0/4151844084 shutdown_connections 2024-02-21T23:53:34.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.274+0000 7f6058d7b640 1 --2- 172.21.15.133:0/4151844084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60540c6f50 0x7f60540ca640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.274+0000 7f6058d7b640 1 --2- 172.21.15.133:0/4151844084 >> v2:172.21.15.133:3300/0 conn(0x7f6054154600 0x7f60540cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.274+0000 7f6058d7b640 1 -- 172.21.15.133:0/4151844084 >> 172.21.15.133:0/4151844084 conn(0x7f6054062000 msgr2=0x7f60540cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:34.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.274+0000 7f6058d7b640 1 -- 172.21.15.133:0/4151844084 shutdown_connections 2024-02-21T23:53:34.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.274+0000 7f6058d7b640 1 -- 172.21.15.133:0/4151844084 wait complete. 2024-02-21T23:53:34.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.276+0000 7f6058d7b640 1 Processor -- start 2024-02-21T23:53:34.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.276+0000 7f6058d7b640 1 -- start start 2024-02-21T23:53:34.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.276+0000 7f6058d7b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60540c6f50 0x7f605406b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f6058d7b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540cab80 0x7f60540cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f6058d7b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6054154600 0x7f60540cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f6058d7b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60540b5720 con 0x7f60540cab80 2024-02-21T23:53:34.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f6058d7b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60540cb860 con 0x7f60540c6f50 2024-02-21T23:53:34.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f6058d7b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60540b58a0 con 0x7f6054154600 2024-02-21T23:53:34.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f605259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60540c6f50 0x7f605406b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f604bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540cab80 0x7f60540cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f6052d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6054154600 0x7f60540cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f605259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60540c6f50 0x7f605406b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58830/0 (socket says 172.21.15.133:58830) 2024-02-21T23:53:34.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.277+0000 7f605259c640 1 -- 172.21.15.133:0/3660766936 learned_addr learned my addr 172.21.15.133:0/3660766936 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:34.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.281+0000 7f605259c640 1 -- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6054154600 msgr2=0x7f60540cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.281+0000 7f605259c640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6054154600 0x7f60540cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.281+0000 7f605259c640 1 -- 172.21.15.133:0/3660766936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540cab80 msgr2=0x7f60540cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.281+0000 7f605259c640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540cab80 0x7f60540cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.281+0000 7f605259c640 1 -- 172.21.15.133:0/3660766936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60540cd920 con 0x7f60540c6f50 2024-02-21T23:53:34.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.281+0000 7f604bfff640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540cab80 0x7f60540cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:53:34.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.283+0000 7f605259c640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60540c6f50 0x7f605406b890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f604400e870 tx=0x7f604400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.287+0000 7f604affd640 1 -- 172.21.15.133:0/3660766936 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044017070 con 0x7f60540c6f50 2024-02-21T23:53:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.287+0000 7f604affd640 1 -- 172.21.15.133:0/3660766936 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604400c940 con 0x7f60540c6f50 2024-02-21T23:53:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.287+0000 7f604affd640 1 -- 172.21.15.133:0/3660766936 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400cd30 con 0x7f60540c6f50 2024-02-21T23:53:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.287+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60540cdc00 con 0x7f60540c6f50 2024-02-21T23:53:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.287+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60540ce1c0 con 0x7f60540c6f50 2024-02-21T23:53:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.288+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60540ca970 con 0x7f60540c6f50 2024-02-21T23:53:34.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.306+0000 7f604affd640 1 -- 172.21.15.133:0/3660766936 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f604401a070 con 0x7f60540c6f50 2024-02-21T23:53:34.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.307+0000 7f604affd640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f603406b520 0x7f603406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.307+0000 7f604bfff640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f603406b520 0x7f603406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.307+0000 7f604bfff640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f603406b520 0x7f603406d9e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f603c036000 tx=0x7f603c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:34.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.311+0000 7f604affd640 1 -- 172.21.15.133:0/3660766936 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6044057290 con 0x7f60540c6f50 2024-02-21T23:53:34.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.311+0000 7f604affd640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f6034075030 0x7f60340774d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.311+0000 7f604affd640 1 -- 172.21.15.133:0/3660766936 --> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6034077ba0 con 0x7f6034075030 2024-02-21T23:53:34.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.311+0000 7f604affd640 1 -- 172.21.15.133:0/3660766936 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6044055490 con 0x7f60540c6f50 2024-02-21T23:53:34.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.311+0000 7f6052d9d640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f6034075030 0x7f60340774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.312+0000 7f6052d9d640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f6034075030 0x7f60340774d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:34.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.312+0000 7f604affd640 1 -- 172.21.15.133:0/3660766936 <== osd.0 v2:172.21.15.133:6800/2680579490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6034077ba0 con 0x7f6034075030 2024-02-21T23:53:34.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.325+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 --> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f605406a130 con 0x7f6034075030 2024-02-21T23:53:34.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.325+0000 7f604affd640 1 -- 172.21.15.133:0/3660766936 <== osd.0 v2:172.21.15.133:6800/2680579490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f605406a130 con 0x7f6034075030 2024-02-21T23:53:34.320 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:53:34.321 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-21T23:53:34.321 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f6034075030 msgr2=0x7f60340774d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.133:6800/2680579490,v1:172.21.15.133:6801/2680579490] conn(0x7f6034075030 0x7f60340774d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f603406b520 msgr2=0x7f603406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f603406b520 0x7f603406d9e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f603c036000 tx=0x7f603c008030 comp rx=0 tx=0).stop 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60540c6f50 msgr2=0x7f605406b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 --2- 172.21.15.133:0/3660766936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60540c6f50 0x7f605406b890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f604400e870 tx=0x7f604400ed30 comp rx=0 tx=0).stop 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6052d9d640 1 -- 172.21.15.133:0/3660766936 reap_dead start 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 shutdown_connections 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 >> 172.21.15.133:0/3660766936 conn(0x7f6054062000 msgr2=0x7f60540c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 shutdown_connections 2024-02-21T23:53:34.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.326+0000 7f6058d7b640 1 -- 172.21.15.133:0/3660766936 wait complete. 2024-02-21T23:53:34.332 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-02-21T23:53:34.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.440+0000 7efcdabc8640 1 Processor -- start 2024-02-21T23:53:34.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.440+0000 7efcdabc8640 1 -- start start 2024-02-21T23:53:34.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.440+0000 7efcdabc8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efcd4154300 0x7efcd406d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.440+0000 7efcdabc8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd415afd0 con 0x7efcd406de00 2024-02-21T23:53:34.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.440+0000 7efcdabc8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd415a170 con 0x7efcd40b0fe0 2024-02-21T23:53:34.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.440+0000 7efcdabc8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd415a450 con 0x7efcd4154300 2024-02-21T23:53:34.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.441+0000 7efcd3fff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7efcd40b0fe0 0x7efcd4072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57476/0 (socket says 172.21.15.133:57476) 2024-02-21T23:53:34.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.441+0000 7efcd3fff640 1 -- 172.21.15.133:0/1203139153 learned_addr learned my addr 172.21.15.133:0/1203139153 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:34.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.441+0000 7efcd8964640 1 --2- 172.21.15.133:0/1203139153 >> v2:172.21.15.133:3300/0 conn(0x7efcd4154300 0x7efcd406d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.444+0000 7efcd37fe640 1 -- 172.21.15.133:0/1203139153 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925909038 0 0) 0x7efcd415a170 con 0x7efcd40b0fe0 2024-02-21T23:53:34.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.444+0000 7efcd37fe640 1 -- 172.21.15.133:0/1203139153 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcc4003520 con 0x7efcd40b0fe0 2024-02-21T23:53:34.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.445+0000 7efcd8964640 1 -- 172.21.15.133:0/1203139153 >> v1:172.21.15.142:6789/0 conn(0x7efcd40b0fe0 legacy=0x7efcd4072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.445+0000 7efcd8964640 1 -- 172.21.15.133:0/1203139153 >> v1:172.21.15.133:6789/0 conn(0x7efcd406de00 legacy=0x7efcd406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.445+0000 7efcd8964640 1 -- 172.21.15.133:0/1203139153 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd415a730 con 0x7efcd4154300 2024-02-21T23:53:34.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.449+0000 7efcd8964640 1 --2- 172.21.15.133:0/1203139153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd4154300 0x7efcd406d0a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcc80097a0 tx=0x7efcc802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec72e2dd23506f8e server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:34.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.453+0000 7efcd37fe640 1 -- 172.21.15.133:0/1203139153 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc802f810 con 0x7efcd4154300 2024-02-21T23:53:34.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.453+0000 7efcd37fe640 1 -- 172.21.15.133:0/1203139153 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc802f9b0 con 0x7efcd4154300 2024-02-21T23:53:34.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.453+0000 7efcd37fe640 1 -- 172.21.15.133:0/1203139153 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc802fd80 con 0x7efcd4154300 2024-02-21T23:53:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.454+0000 7efcdabc8640 1 -- 172.21.15.133:0/1203139153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd4154300 msgr2=0x7efcd406d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.454+0000 7efcdabc8640 1 --2- 172.21.15.133:0/1203139153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd4154300 0x7efcd406d0a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcc80097a0 tx=0x7efcc802b9f0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.454+0000 7efcdabc8640 1 -- 172.21.15.133:0/1203139153 shutdown_connections 2024-02-21T23:53:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.454+0000 7efcdabc8640 1 --2- 172.21.15.133:0/1203139153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd4154300 0x7efcd406d0a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.454+0000 7efcdabc8640 1 -- 172.21.15.133:0/1203139153 >> 172.21.15.133:0/1203139153 conn(0x7efcd40cc620 msgr2=0x7efcd40c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.454+0000 7efcdabc8640 1 -- 172.21.15.133:0/1203139153 shutdown_connections 2024-02-21T23:53:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.454+0000 7efcdabc8640 1 -- 172.21.15.133:0/1203139153 wait complete. 2024-02-21T23:53:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.455+0000 7efcdabc8640 1 Processor -- start 2024-02-21T23:53:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.455+0000 7efcdabc8640 1 -- start start 2024-02-21T23:53:34.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.456+0000 7efcdabc8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcd406de00 0x7efcd4159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.456+0000 7efcdabc8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd40b0fe0 0x7efcd40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.456+0000 7efcdabc8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcd4154300 0x7efcd414b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.456+0000 7efcdabc8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd41546e0 con 0x7efcd40b0fe0 2024-02-21T23:53:34.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.456+0000 7efcdabc8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd415acf0 con 0x7efcd406de00 2024-02-21T23:53:34.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.456+0000 7efcdabc8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd406d5e0 con 0x7efcd4154300 2024-02-21T23:53:34.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.457+0000 7efcd8964640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcd406de00 0x7efcd4159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.457+0000 7efcd3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd40b0fe0 0x7efcd40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.457+0000 7efcd9165640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcd4154300 0x7efcd414b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.457+0000 7efcd8964640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcd406de00 0x7efcd4159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58846/0 (socket says 172.21.15.133:58846) 2024-02-21T23:53:34.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.457+0000 7efcd8964640 1 -- 172.21.15.133:0/3275245844 learned_addr learned my addr 172.21.15.133:0/3275245844 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:34.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.460+0000 7efcd3fff640 1 -- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcd4154300 msgr2=0x7efcd414b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.460+0000 7efcd3fff640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcd4154300 0x7efcd414b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.460+0000 7efcd3fff640 1 -- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcd406de00 msgr2=0x7efcd4159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.460+0000 7efcd3fff640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcd406de00 0x7efcd4159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.460+0000 7efcd3fff640 1 -- 172.21.15.133:0/3275245844 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4148f30 con 0x7efcd40b0fe0 2024-02-21T23:53:34.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.463+0000 7efcd3fff640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd40b0fe0 0x7efcd40cba20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efcc000c430 tx=0x7efcc000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:34.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.466+0000 7efcd17fa640 1 -- 172.21.15.133:0/3275245844 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0017070 con 0x7efcd40b0fe0 2024-02-21T23:53:34.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.466+0000 7efcd17fa640 1 -- 172.21.15.133:0/3275245844 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc0005ce0 con 0x7efcd40b0fe0 2024-02-21T23:53:34.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.466+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd40cb270 con 0x7efcd40b0fe0 2024-02-21T23:53:34.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.466+0000 7efcd17fa640 1 -- 172.21.15.133:0/3275245844 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00072d0 con 0x7efcd40b0fe0 2024-02-21T23:53:34.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.466+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd41f07e0 con 0x7efcd40b0fe0 2024-02-21T23:53:34.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.467+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd406a130 con 0x7efcd40b0fe0 2024-02-21T23:53:34.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.487+0000 7efcd17fa640 1 -- 172.21.15.133:0/3275245844 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcc001a070 con 0x7efcd40b0fe0 2024-02-21T23:53:34.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.487+0000 7efcd17fa640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efcb806b520 0x7efcb806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:34.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.487+0000 7efcd8964640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efcb806b520 0x7efcb806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:34.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.488+0000 7efcd8964640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efcb806b520 0x7efcb806d9e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009560 tx=0x7efcc8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.723+0000 7efcd17fa640 1 -- 172.21.15.133:0/3275245844 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7efcc0056310 con 0x7efcd40b0fe0 2024-02-21T23:53:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.723+0000 7efcd17fa640 1 -- 172.21.15.133:0/3275245844 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7efcc0058270 con 0x7efcd40b0fe0 2024-02-21T23:53:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.902+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7efcd406c880 con 0x7efcb806b520 2024-02-21T23:53:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.903+0000 7efcd17fa640 1 -- 172.21.15.133:0/3275245844 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efcd406c880 con 0x7efcb806b520 2024-02-21T23:53:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 0 to deep-scrub 2024-02-21T23:53:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efcb806b520 msgr2=0x7efcb806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efcb806b520 0x7efcb806d9e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009560 tx=0x7efcc8008030 comp rx=0 tx=0).stop 2024-02-21T23:53:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd40b0fe0 msgr2=0x7efcd40cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd40b0fe0 0x7efcd40cba20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efcc000c430 tx=0x7efcc000c8f0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 shutdown_connections 2024-02-21T23:53:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efcb806b520 0x7efcb806d9e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcd4154300 0x7efcd414b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcd40b0fe0 0x7efcd40cba20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 --2- 172.21.15.133:0/3275245844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcd406de00 0x7efcd4159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 >> 172.21.15.133:0/3275245844 conn(0x7efcd40cc620 msgr2=0x7efcd40743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 shutdown_connections 2024-02-21T23:53:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:34.905+0000 7efcdabc8640 1 -- 172.21.15.133:0/3275245844 wait complete. 2024-02-21T23:53:34.911 INFO:tasks.ceph:Scrubbing osd.1 2024-02-21T23:53:34.911 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-02-21T23:53:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.020+0000 7fb38c288640 1 Processor -- start 2024-02-21T23:53:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.020+0000 7fb38c288640 1 -- start start 2024-02-21T23:53:35.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.021+0000 7fb38c288640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3841542e0 0x7fb38415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.021+0000 7fb38c288640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38406c3e0 con 0x7fb3840aff30 2024-02-21T23:53:35.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.021+0000 7fb38c288640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38406cc40 con 0x7fb38406bfd0 2024-02-21T23:53:35.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.021+0000 7fb38c288640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3840b05a0 con 0x7fb3841542e0 2024-02-21T23:53:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.022+0000 7fb38a825640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fb38406bfd0 0x7fb384164b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57482/0 (socket says 172.21.15.133:57482) 2024-02-21T23:53:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.022+0000 7fb38a825640 1 -- 172.21.15.133:0/2684629959 learned_addr learned my addr 172.21.15.133:0/2684629959 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.022+0000 7fb38a024640 1 --2- 172.21.15.133:0/2684629959 >> v2:172.21.15.133:3300/0 conn(0x7fb3841542e0 0x7fb38415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.024+0000 7fb389022640 1 -- 172.21.15.133:0/2684629959 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502675202 0 0) 0x7fb38406cc40 con 0x7fb38406bfd0 2024-02-21T23:53:35.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.024+0000 7fb389022640 1 -- 172.21.15.133:0/2684629959 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb370003520 con 0x7fb38406bfd0 2024-02-21T23:53:35.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.025+0000 7fb38a024640 1 -- 172.21.15.133:0/2684629959 >> v1:172.21.15.142:6789/0 conn(0x7fb38406bfd0 legacy=0x7fb384164b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.025+0000 7fb38a024640 1 -- 172.21.15.133:0/2684629959 >> v1:172.21.15.133:6789/0 conn(0x7fb3840aff30 legacy=0x7fb3840cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.025+0000 7fb38a024640 1 -- 172.21.15.133:0/2684629959 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38406c680 con 0x7fb3841542e0 2024-02-21T23:53:35.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.029+0000 7fb38a024640 1 --2- 172.21.15.133:0/2684629959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841542e0 0x7fb38415e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb380009d90 tx=0x7fb38002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea81c690b22319f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.033+0000 7fb389022640 1 -- 172.21.15.133:0/2684629959 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380039070 con 0x7fb3841542e0 2024-02-21T23:53:35.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.033+0000 7fb389022640 1 -- 172.21.15.133:0/2684629959 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38002cbd0 con 0x7fb3841542e0 2024-02-21T23:53:35.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.033+0000 7fb389022640 1 -- 172.21.15.133:0/2684629959 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380034440 con 0x7fb3841542e0 2024-02-21T23:53:35.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.034+0000 7fb38c288640 1 -- 172.21.15.133:0/2684629959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841542e0 msgr2=0x7fb38415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.034+0000 7fb38c288640 1 --2- 172.21.15.133:0/2684629959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841542e0 0x7fb38415e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb380009d90 tx=0x7fb38002be20 comp rx=0 tx=0).stop 2024-02-21T23:53:35.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.034+0000 7fb38c288640 1 -- 172.21.15.133:0/2684629959 shutdown_connections 2024-02-21T23:53:35.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.034+0000 7fb38c288640 1 --2- 172.21.15.133:0/2684629959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841542e0 0x7fb38415e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.034+0000 7fb38c288640 1 -- 172.21.15.133:0/2684629959 >> 172.21.15.133:0/2684629959 conn(0x7fb38406aec0 msgr2=0x7fb38406f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.034+0000 7fb38c288640 1 -- 172.21.15.133:0/2684629959 shutdown_connections 2024-02-21T23:53:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.034+0000 7fb38c288640 1 -- 172.21.15.133:0/2684629959 wait complete. 2024-02-21T23:53:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.035+0000 7fb38c288640 1 Processor -- start 2024-02-21T23:53:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.035+0000 7fb38c288640 1 -- start start 2024-02-21T23:53:35.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.036+0000 7fb38c288640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38406bfd0 0x7fb3840add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.036+0000 7fb38c288640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3840aff30 0x7fb3841f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.036+0000 7fb38c288640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3841542e0 0x7fb3841f0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.036+0000 7fb38c288640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38415daf0 con 0x7fb38406bfd0 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.036+0000 7fb38c288640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38415d810 con 0x7fb3841542e0 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.036+0000 7fb38c288640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38415ddd0 con 0x7fb3840aff30 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.036+0000 7fb38a024640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38406bfd0 0x7fb3840add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.036+0000 7fb38a825640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3841542e0 0x7fb3841f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.037+0000 7fb389823640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3840aff30 0x7fb3841f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.037+0000 7fb38a024640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38406bfd0 0x7fb3840add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36218/0 (socket says 172.21.15.133:36218) 2024-02-21T23:53:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.037+0000 7fb38a024640 1 -- 172.21.15.133:0/1360265970 learned_addr learned my addr 172.21.15.133:0/1360265970 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.040+0000 7fb38a024640 1 -- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3840aff30 msgr2=0x7fb3841f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.040+0000 7fb38a024640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3840aff30 0x7fb3841f02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.040+0000 7fb38a024640 1 -- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3841542e0 msgr2=0x7fb3841f0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.040+0000 7fb38a024640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3841542e0 0x7fb3841f0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.040+0000 7fb38a024640 1 -- 172.21.15.133:0/1360265970 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840c91a0 con 0x7fb38406bfd0 2024-02-21T23:53:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.040+0000 7fb38a825640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3841542e0 0x7fb3841f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:53:35.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.042+0000 7fb38a024640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38406bfd0 0x7fb3840add20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb380037040 tx=0x7fb380004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.062+0000 7fb37affd640 1 -- 172.21.15.133:0/1360265970 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380039050 con 0x7fb38406bfd0 2024-02-21T23:53:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.062+0000 7fb37affd640 1 -- 172.21.15.133:0/1360265970 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb380045070 con 0x7fb38406bfd0 2024-02-21T23:53:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.062+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3840c3250 con 0x7fb38406bfd0 2024-02-21T23:53:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.062+0000 7fb37affd640 1 -- 172.21.15.133:0/1360265970 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38003e850 con 0x7fb38406bfd0 2024-02-21T23:53:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.062+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb384153460 con 0x7fb38406bfd0 2024-02-21T23:53:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.063+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb38406a130 con 0x7fb38406bfd0 2024-02-21T23:53:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.080+0000 7fb37affd640 1 -- 172.21.15.133:0/1360265970 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb38003e9f0 con 0x7fb38406bfd0 2024-02-21T23:53:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.080+0000 7fb37affd640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb36806b550 0x7fb36806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.081+0000 7fb389823640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb36806b550 0x7fb36806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.081+0000 7fb389823640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb36806b550 0x7fb36806da10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb38414e330 tx=0x7fb374009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.084+0000 7fb37affd640 1 -- 172.21.15.133:0/1360265970 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fb380079e60 con 0x7fb38406bfd0 2024-02-21T23:53:35.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.084+0000 7fb37affd640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fb368075060 0x7fb368077500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.084+0000 7fb37affd640 1 -- 172.21.15.133:0/1360265970 --> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb380044ec0 con 0x7fb368075060 2024-02-21T23:53:35.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.084+0000 7fb38a825640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fb368075060 0x7fb368077500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.084+0000 7fb38a825640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fb368075060 0x7fb368077500 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.085+0000 7fb37affd640 1 -- 172.21.15.133:0/1360265970 <== osd.1 v2:172.21.15.133:6808/198853132 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb380044ec0 con 0x7fb368075060 2024-02-21T23:53:35.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.088+0000 7fb37affd640 1 -- 172.21.15.133:0/1360265970 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb380078060 con 0x7fb38406bfd0 2024-02-21T23:53:35.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.098+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 --> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb3840af6e0 con 0x7fb368075060 2024-02-21T23:53:35.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.098+0000 7fb37affd640 1 -- 172.21.15.133:0/1360265970 <== osd.1 v2:172.21.15.133:6808/198853132 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb3840af6e0 con 0x7fb368075060 2024-02-21T23:53:35.094 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:53:35.094 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-21T23:53:35.094 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:53:35.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.098+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fb368075060 msgr2=0x7fb368077500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.098+0000 7fb38c288640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.133:6808/198853132,v1:172.21.15.133:6809/198853132] conn(0x7fb368075060 0x7fb368077500 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.098+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb36806b550 msgr2=0x7fb36806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.098+0000 7fb38c288640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb36806b550 0x7fb36806da10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb38414e330 tx=0x7fb374009040 comp rx=0 tx=0).stop 2024-02-21T23:53:35.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.098+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38406bfd0 msgr2=0x7fb3840add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.098+0000 7fb38c288640 1 --2- 172.21.15.133:0/1360265970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38406bfd0 0x7fb3840add20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb380037040 tx=0x7fb380004a40 comp rx=0 tx=0).stop 2024-02-21T23:53:35.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.098+0000 7fb38a825640 1 -- 172.21.15.133:0/1360265970 reap_dead start 2024-02-21T23:53:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.099+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 shutdown_connections 2024-02-21T23:53:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.099+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 >> 172.21.15.133:0/1360265970 conn(0x7fb38406aec0 msgr2=0x7fb38414f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.099+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 shutdown_connections 2024-02-21T23:53:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.099+0000 7fb38c288640 1 -- 172.21.15.133:0/1360265970 wait complete. 2024-02-21T23:53:35.105 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-02-21T23:53:35.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.213+0000 7f53b37fa640 1 Processor -- start 2024-02-21T23:53:35.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.213+0000 7f53b37fa640 1 -- start start 2024-02-21T23:53:35.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.214+0000 7f53b37fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53ac150f80 0x7f53ac15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.214+0000 7f53b37fa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53ac15b020 0x7f53ac0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.214+0000 7f53b37fa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f53ac0cbb30 0x7f53ac0c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.214+0000 7f53b37fa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ac0c2b40 con 0x7f53ac15b020 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.214+0000 7f53b37fa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ac0c2d70 con 0x7f53ac0cbb30 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.214+0000 7f53b37fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0c3050 con 0x7f53ac150f80 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.215+0000 7f53b0d95640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53ac15b020 0x7f53ac0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.215+0000 7f53b0d95640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53ac15b020 0x7f53ac0cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36230/0 (socket says 172.21.15.133:36230) 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.215+0000 7f53b0d95640 1 -- 172.21.15.133:0/4146688669 learned_addr learned my addr 172.21.15.133:0/4146688669 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.217+0000 7f53b0d95640 1 -- 172.21.15.133:0/4146688669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53ac150f80 msgr2=0x7f53ac15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.217+0000 7f53b0d95640 1 --2- 172.21.15.133:0/4146688669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53ac150f80 0x7f53ac15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.217+0000 7f53b0d95640 1 -- 172.21.15.133:0/4146688669 >> v2:172.21.15.142:3300/0 conn(0x7f53ac0cbb30 msgr2=0x7f53ac0c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.217+0000 7f53b0d95640 1 --2- 172.21.15.133:0/4146688669 >> v2:172.21.15.142:3300/0 conn(0x7f53ac0cbb30 0x7f53ac0c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.217+0000 7f53b0d95640 1 -- 172.21.15.133:0/4146688669 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac0c3330 con 0x7f53ac15b020 2024-02-21T23:53:35.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.219+0000 7f53b0d95640 1 --2- 172.21.15.133:0/4146688669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 0x7f53ac0cb5f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0c94d0 tx=0x7f53a802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d90bf911ddf348a server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.223+0000 7f53a3fff640 1 -- 172.21.15.133:0/4146688669 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8034070 con 0x7f53ac15b020 2024-02-21T23:53:35.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.223+0000 7f53a3fff640 1 -- 172.21.15.133:0/4146688669 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a8030860 con 0x7f53ac15b020 2024-02-21T23:53:35.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.223+0000 7f53a3fff640 1 -- 172.21.15.133:0/4146688669 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8030c30 con 0x7f53ac15b020 2024-02-21T23:53:35.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.223+0000 7f53b37fa640 1 -- 172.21.15.133:0/4146688669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 msgr2=0x7f53ac0cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.223+0000 7f53b37fa640 1 --2- 172.21.15.133:0/4146688669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 0x7f53ac0cb5f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0c94d0 tx=0x7f53a802b9a0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.224+0000 7f53b37fa640 1 -- 172.21.15.133:0/4146688669 shutdown_connections 2024-02-21T23:53:35.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.224+0000 7f53b37fa640 1 --2- 172.21.15.133:0/4146688669 >> v2:172.21.15.142:3300/0 conn(0x7f53ac0cbb30 0x7f53ac0c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.224+0000 7f53b37fa640 1 --2- 172.21.15.133:0/4146688669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 0x7f53ac0cb5f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.224+0000 7f53b37fa640 1 --2- 172.21.15.133:0/4146688669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53ac150f80 0x7f53ac15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.224+0000 7f53b37fa640 1 -- 172.21.15.133:0/4146688669 >> 172.21.15.133:0/4146688669 conn(0x7f53ac06aec0 msgr2=0x7f53ac06b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:35.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.224+0000 7f53b37fa640 1 -- 172.21.15.133:0/4146688669 shutdown_connections 2024-02-21T23:53:35.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.224+0000 7f53b37fa640 1 -- 172.21.15.133:0/4146688669 wait complete. 2024-02-21T23:53:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.225+0000 7f53b37fa640 1 Processor -- start 2024-02-21T23:53:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.225+0000 7f53b37fa640 1 -- start start 2024-02-21T23:53:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.225+0000 7f53b37fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ac0cbb30 0x7f53ac1609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.225+0000 7f53b37fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53ac150f80 0x7f53ac14c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.225+0000 7f53b37fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 0x7f53ac158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.225+0000 7f53b37fa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0c5990 con 0x7f53ac15b020 2024-02-21T23:53:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.225+0000 7f53b37fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac15a5c0 con 0x7f53ac0cbb30 2024-02-21T23:53:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.225+0000 7f53b37fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0c5b10 con 0x7f53ac150f80 2024-02-21T23:53:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.226+0000 7f53b1d97640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 0x7f53ac158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.226+0000 7f53b1d97640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 0x7f53ac158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36240/0 (socket says 172.21.15.133:36240) 2024-02-21T23:53:35.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.226+0000 7f53b1d97640 1 -- 172.21.15.133:0/2665777617 learned_addr learned my addr 172.21.15.133:0/2665777617 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:35.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.227+0000 7f53b1596640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ac0cbb30 0x7f53ac1609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.229+0000 7f53b1d97640 1 -- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53ac150f80 msgr2=0x7f53ac14c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.229+0000 7f53b1d97640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53ac150f80 0x7f53ac14c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.229+0000 7f53b1d97640 1 -- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ac0cbb30 msgr2=0x7f53ac1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.229+0000 7f53b1d97640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ac0cbb30 0x7f53ac1609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.229+0000 7f53b1d97640 1 -- 172.21.15.133:0/2665777617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac15a2b0 con 0x7f53ac15b020 2024-02-21T23:53:35.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.233+0000 7f53b1d97640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 0x7f53ac158f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f53a400b9a0 tx=0x7f53a400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.239+0000 7f53a1ffb640 1 -- 172.21.15.133:0/2665777617 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4019070 con 0x7f53ac15b020 2024-02-21T23:53:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.239+0000 7f53a1ffb640 1 -- 172.21.15.133:0/2665777617 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a400cac0 con 0x7f53ac15b020 2024-02-21T23:53:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.239+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac15f940 con 0x7f53ac15b020 2024-02-21T23:53:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.239+0000 7f53a1ffb640 1 -- 172.21.15.133:0/2665777617 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4014440 con 0x7f53ac15b020 2024-02-21T23:53:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.239+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac15fec0 con 0x7f53ac15b020 2024-02-21T23:53:35.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.240+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac0c4660 con 0x7f53ac15b020 2024-02-21T23:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.260+0000 7f53a1ffb640 1 -- 172.21.15.133:0/2665777617 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53a401c070 con 0x7f53ac15b020 2024-02-21T23:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.260+0000 7f53a1ffb640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f539006b520 0x7f539006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.260+0000 7f53b1596640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f539006b520 0x7f539006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.261+0000 7f53b1596640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f539006b520 0x7f539006d9e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f539c009990 tx=0x7f539c009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.263+0000 7f53a1ffb640 1 -- 172.21.15.133:0/2665777617 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f53a4056b20 con 0x7f53ac15b020 2024-02-21T23:53:35.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.502+0000 7f53a1ffb640 1 -- 172.21.15.133:0/2665777617 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f53ac0c4660 con 0x7f53ac15b020 2024-02-21T23:53:35.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.678+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f53ac14e570 con 0x7f539006b520 2024-02-21T23:53:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.679+0000 7f53a1ffb640 1 -- 172.21.15.133:0/2665777617 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f53ac14e570 con 0x7f539006b520 2024-02-21T23:53:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 1 to deep-scrub 2024-02-21T23:53:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f539006b520 msgr2=0x7f539006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f539006b520 0x7f539006d9e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f539c009990 tx=0x7f539c009110 comp rx=0 tx=0).stop 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 msgr2=0x7f53ac158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 0x7f53ac158f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f53a400b9a0 tx=0x7f53a400bda0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 shutdown_connections 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f539006b520 0x7f539006d9e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53ac15b020 0x7f53ac158f40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53ac150f80 0x7f53ac14c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 --2- 172.21.15.133:0/2665777617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ac0cbb30 0x7f53ac1609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 >> 172.21.15.133:0/2665777617 conn(0x7f53ac06aec0 msgr2=0x7f53ac1436c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 shutdown_connections 2024-02-21T23:53:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.681+0000 7f53b37fa640 1 -- 172.21.15.133:0/2665777617 wait complete. 2024-02-21T23:53:35.687 INFO:tasks.ceph:Scrubbing osd.2 2024-02-21T23:53:35.687 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-02-21T23:53:35.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.797+0000 7f346ae9d640 1 Processor -- start 2024-02-21T23:53:35.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.797+0000 7f346ae9d640 1 -- start start 2024-02-21T23:53:35.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.797+0000 7f346ae9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3464154300 0x7f346406d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.797+0000 7f346ae9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640b0fe0 0x7f3464072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.797+0000 7f346ae9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3464072f80 0x7f346406de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.797+0000 7f346ae9d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346415afd0 con 0x7f3464072f80 2024-02-21T23:53:35.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.797+0000 7f346ae9d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346415a170 con 0x7f3464154300 2024-02-21T23:53:35.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.797+0000 7f346ae9d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346415a450 con 0x7f34640b0fe0 2024-02-21T23:53:35.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.798+0000 7f346943a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3464072f80 0x7f346406de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.798+0000 7f3468c39640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3464154300 0x7f346406d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.798+0000 7f3463fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640b0fe0 0x7f3464072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.798+0000 7f3468c39640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3464154300 0x7f346406d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58888/0 (socket says 172.21.15.133:58888) 2024-02-21T23:53:35.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.798+0000 7f3468c39640 1 -- 172.21.15.133:0/1453748621 learned_addr learned my addr 172.21.15.133:0/1453748621 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:35.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.800+0000 7f3468c39640 1 -- 172.21.15.133:0/1453748621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640b0fe0 msgr2=0x7f3464072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:35.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.800+0000 7f3468c39640 1 --2- 172.21.15.133:0/1453748621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640b0fe0 0x7f3464072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.800+0000 7f3468c39640 1 -- 172.21.15.133:0/1453748621 >> v2:172.21.15.133:3300/0 conn(0x7f3464072f80 msgr2=0x7f346406de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:35.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.800+0000 7f3468c39640 1 --2- 172.21.15.133:0/1453748621 >> v2:172.21.15.133:3300/0 conn(0x7f3464072f80 0x7f346406de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.800+0000 7f3468c39640 1 -- 172.21.15.133:0/1453748621 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346415a730 con 0x7f3464154300 2024-02-21T23:53:35.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.802+0000 7f3468c39640 1 --2- 172.21.15.133:0/1453748621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464154300 0x7f346406d0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f34640c66b0 tx=0x7f345802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a282ef6cfa60b065 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.806+0000 7f34637fe640 1 -- 172.21.15.133:0/1453748621 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458034070 con 0x7f3464154300 2024-02-21T23:53:35.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.806+0000 7f34637fe640 1 -- 172.21.15.133:0/1453748621 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3458030860 con 0x7f3464154300 2024-02-21T23:53:35.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.806+0000 7f34637fe640 1 -- 172.21.15.133:0/1453748621 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458030c30 con 0x7f3464154300 2024-02-21T23:53:35.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.807+0000 7f346ae9d640 1 -- 172.21.15.133:0/1453748621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464154300 msgr2=0x7f346406d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.807+0000 7f346ae9d640 1 --2- 172.21.15.133:0/1453748621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464154300 0x7f346406d0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f34640c66b0 tx=0x7f345802b9a0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.807+0000 7f346ae9d640 1 -- 172.21.15.133:0/1453748621 shutdown_connections 2024-02-21T23:53:35.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.807+0000 7f346ae9d640 1 --2- 172.21.15.133:0/1453748621 >> v2:172.21.15.133:3300/0 conn(0x7f3464072f80 0x7f346406de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.807+0000 7f346ae9d640 1 --2- 172.21.15.133:0/1453748621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640b0fe0 0x7f3464072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.807+0000 7f346ae9d640 1 --2- 172.21.15.133:0/1453748621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464154300 0x7f346406d0a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.807+0000 7f346ae9d640 1 -- 172.21.15.133:0/1453748621 >> 172.21.15.133:0/1453748621 conn(0x7f34640cc620 msgr2=0x7f34640c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:35.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.807+0000 7f346ae9d640 1 -- 172.21.15.133:0/1453748621 shutdown_connections 2024-02-21T23:53:35.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.807+0000 7f346ae9d640 1 -- 172.21.15.133:0/1453748621 wait complete. 2024-02-21T23:53:35.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.808+0000 7f346ae9d640 1 Processor -- start 2024-02-21T23:53:35.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.808+0000 7f346ae9d640 1 -- start start 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.809+0000 7f346ae9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464072f80 0x7f3464071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.809+0000 7f346ae9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34640b0fe0 0x7f3464153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.809+0000 7f346ae9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3464154300 0x7f34641effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.809+0000 7f346ae9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34641546e0 con 0x7f34640b0fe0 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.809+0000 7f346ae9d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346415acf0 con 0x7f3464072f80 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.809+0000 7f346ae9d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346406d5e0 con 0x7f3464154300 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.809+0000 7f3468c39640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464072f80 0x7f3464071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.809+0000 7f3463fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34640b0fe0 0x7f3464153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.810+0000 7f346943a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3464154300 0x7f34641effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.810+0000 7f3468c39640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464072f80 0x7f3464071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58892/0 (socket says 172.21.15.133:58892) 2024-02-21T23:53:35.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.810+0000 7f3468c39640 1 -- 172.21.15.133:0/3543518970 learned_addr learned my addr 172.21.15.133:0/3543518970 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:35.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.813+0000 7f3468c39640 1 -- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3464154300 msgr2=0x7f34641effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.813+0000 7f3468c39640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3464154300 0x7f34641effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.813+0000 7f3468c39640 1 -- 172.21.15.133:0/3543518970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34640b0fe0 msgr2=0x7f3464153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.813+0000 7f3468c39640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34640b0fe0 0x7f3464153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.813+0000 7f3468c39640 1 -- 172.21.15.133:0/3543518970 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464149070 con 0x7f3464072f80 2024-02-21T23:53:35.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.815+0000 7f3468c39640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464072f80 0x7f3464071ca0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3458009960 tx=0x7f34580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.818+0000 7f34617fa640 1 -- 172.21.15.133:0/3543518970 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458034040 con 0x7f3464072f80 2024-02-21T23:53:35.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.818+0000 7f34617fa640 1 -- 172.21.15.133:0/3543518970 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3458039d90 con 0x7f3464072f80 2024-02-21T23:53:35.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.819+0000 7f34617fa640 1 -- 172.21.15.133:0/3543518970 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580433f0 con 0x7f3464072f80 2024-02-21T23:53:35.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.819+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3464149350 con 0x7f3464072f80 2024-02-21T23:53:35.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.819+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3464149910 con 0x7f3464072f80 2024-02-21T23:53:35.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.819+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f346406a130 con 0x7f3464072f80 2024-02-21T23:53:35.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.838+0000 7f34617fa640 1 -- 172.21.15.133:0/3543518970 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3458037070 con 0x7f3464072f80 2024-02-21T23:53:35.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.839+0000 7f34617fa640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f344806b550 0x7f344806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.839+0000 7f3463fff640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f344806b550 0x7f344806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.839+0000 7f3463fff640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f344806b550 0x7f344806da10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f346406f050 tx=0x7f3450009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.841+0000 7f34617fa640 1 -- 172.21.15.133:0/3543518970 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f3458079510 con 0x7f3464072f80 2024-02-21T23:53:35.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.842+0000 7f34617fa640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f3448075070 0x7f3448077510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.842+0000 7f34617fa640 1 -- 172.21.15.133:0/3543518970 --> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f345803ce50 con 0x7f3448075070 2024-02-21T23:53:35.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.842+0000 7f34617fa640 1 -- 172.21.15.133:0/3543518970 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3458077710 con 0x7f3464072f80 2024-02-21T23:53:35.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.842+0000 7f346943a640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f3448075070 0x7f3448077510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.842+0000 7f346943a640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f3448075070 0x7f3448077510 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.843+0000 7f34617fa640 1 -- 172.21.15.133:0/3543518970 <== osd.2 v2:172.21.15.133:6816/3824311867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f345803ce50 con 0x7f3448075070 2024-02-21T23:53:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.855+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 --> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3464149b50 con 0x7f3448075070 2024-02-21T23:53:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f34617fa640 1 -- 172.21.15.133:0/3543518970 <== osd.2 v2:172.21.15.133:6816/3824311867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3464149b50 con 0x7f3448075070 2024-02-21T23:53:35.851 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:53:35.851 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-21T23:53:35.851 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:53:35.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f3448075070 msgr2=0x7f3448077510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.133:6816/3824311867,v1:172.21.15.133:6817/3824311867] conn(0x7f3448075070 0x7f3448077510 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f344806b550 msgr2=0x7f344806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f344806b550 0x7f344806da10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f346406f050 tx=0x7f3450009190 comp rx=0 tx=0).stop 2024-02-21T23:53:35.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464072f80 msgr2=0x7f3464071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 --2- 172.21.15.133:0/3543518970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464072f80 0x7f3464071ca0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3458009960 tx=0x7f34580046c0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346943a640 1 -- 172.21.15.133:0/3543518970 reap_dead start 2024-02-21T23:53:35.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 shutdown_connections 2024-02-21T23:53:35.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 >> 172.21.15.133:0/3543518970 conn(0x7f34640cc620 msgr2=0x7f346414fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:35.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 shutdown_connections 2024-02-21T23:53:35.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.856+0000 7f346ae9d640 1 -- 172.21.15.133:0/3543518970 wait complete. 2024-02-21T23:53:35.862 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-02-21T23:53:35.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.970+0000 7f04cb2de640 1 Processor -- start 2024-02-21T23:53:35.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.970+0000 7f04cb2de640 1 -- start start 2024-02-21T23:53:35.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.971+0000 7f04cb2de640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f04c406de00 0x7f04c406e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.971+0000 7f04cb2de640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c415afd0 con 0x7f04c40b0fe0 2024-02-21T23:53:35.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.971+0000 7f04cb2de640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c415a170 con 0x7f04c4154300 2024-02-21T23:53:35.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.971+0000 7f04cb2de640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c415a450 con 0x7f04c406de00 2024-02-21T23:53:35.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.971+0000 7f04c907a640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f04c4154300 0x7f04c406d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57488/0 (socket says 172.21.15.133:57488) 2024-02-21T23:53:35.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.971+0000 7f04c907a640 1 -- 172.21.15.133:0/1778624578 learned_addr learned my addr 172.21.15.133:0/1778624578 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:35.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.971+0000 7f04c987b640 1 --2- 172.21.15.133:0/1778624578 >> v2:172.21.15.133:3300/0 conn(0x7f04c406de00 0x7f04c406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.973+0000 7f04bbfff640 1 -- 172.21.15.133:0/1778624578 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336017143 0 0) 0x7f04c415a170 con 0x7f04c4154300 2024-02-21T23:53:35.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.973+0000 7f04bbfff640 1 -- 172.21.15.133:0/1778624578 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04b0003520 con 0x7f04c4154300 2024-02-21T23:53:35.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.976+0000 7f04c987b640 1 -- 172.21.15.133:0/1778624578 >> v1:172.21.15.142:6789/0 conn(0x7f04c4154300 legacy=0x7f04c406d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.976+0000 7f04c987b640 1 -- 172.21.15.133:0/1778624578 >> v1:172.21.15.133:6789/0 conn(0x7f04c40b0fe0 legacy=0x7f04c4072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.976+0000 7f04c987b640 1 -- 172.21.15.133:0/1778624578 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c415a730 con 0x7f04c406de00 2024-02-21T23:53:35.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.979+0000 7f04c987b640 1 --2- 172.21.15.133:0/1778624578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 0x7f04c406e210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009d90 tx=0x7f04bc02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1785394bf90dfd6 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.983+0000 7f04bbfff640 1 -- 172.21.15.133:0/1778624578 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc039070 con 0x7f04c406de00 2024-02-21T23:53:35.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.983+0000 7f04bbfff640 1 -- 172.21.15.133:0/1778624578 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc02cd80 con 0x7f04c406de00 2024-02-21T23:53:35.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.983+0000 7f04bbfff640 1 -- 172.21.15.133:0/1778624578 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc034660 con 0x7f04c406de00 2024-02-21T23:53:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.983+0000 7f04cb2de640 1 -- 172.21.15.133:0/1778624578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 msgr2=0x7f04c406e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.983+0000 7f04cb2de640 1 --2- 172.21.15.133:0/1778624578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 0x7f04c406e210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009d90 tx=0x7f04bc02bfd0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.983+0000 7f04cb2de640 1 -- 172.21.15.133:0/1778624578 shutdown_connections 2024-02-21T23:53:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.983+0000 7f04cb2de640 1 --2- 172.21.15.133:0/1778624578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 0x7f04c406e210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.983+0000 7f04cb2de640 1 -- 172.21.15.133:0/1778624578 >> 172.21.15.133:0/1778624578 conn(0x7f04c40cc620 msgr2=0x7f04c40c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.984+0000 7f04cb2de640 1 -- 172.21.15.133:0/1778624578 shutdown_connections 2024-02-21T23:53:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.984+0000 7f04cb2de640 1 -- 172.21.15.133:0/1778624578 wait complete. 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.985+0000 7f04cb2de640 1 Processor -- start 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.985+0000 7f04cb2de640 1 -- start start 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.985+0000 7f04cb2de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 0x7f04c414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.985+0000 7f04cb2de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04c40b0fe0 0x7f04c41e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.985+0000 7f04cb2de640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c4154300 0x7f04c414c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.985+0000 7f04cb2de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c406e750 con 0x7f04c406de00 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.985+0000 7f04cb2de640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c415acf0 con 0x7f04c4154300 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.985+0000 7f04cb2de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41546e0 con 0x7f04c40b0fe0 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.986+0000 7f04c907a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 0x7f04c414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.986+0000 7f04c987b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c4154300 0x7f04c414c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.986+0000 7f04c987b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c4154300 0x7f04c414c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58906/0 (socket says 172.21.15.133:58906) 2024-02-21T23:53:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.986+0000 7f04c987b640 1 -- 172.21.15.133:0/2269630473 learned_addr learned my addr 172.21.15.133:0/2269630473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:35.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.988+0000 7f04c8879640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04c40b0fe0 0x7f04c41e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.989+0000 7f04c907a640 1 -- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04c40b0fe0 msgr2=0x7f04c41e5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.989+0000 7f04c907a640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04c40b0fe0 0x7f04c41e5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.989+0000 7f04c907a640 1 -- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c4154300 msgr2=0x7f04c414c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.989+0000 7f04c907a640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c4154300 0x7f04c414c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.989+0000 7f04c907a640 1 -- 172.21.15.133:0/2269630473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c41f2dc0 con 0x7f04c406de00 2024-02-21T23:53:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.991+0000 7f04c907a640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 0x7f04c414ba70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04c000c3f0 tx=0x7f04c0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.995+0000 7f04b9ffb640 1 -- 172.21.15.133:0/2269630473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0017070 con 0x7f04c406de00 2024-02-21T23:53:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.995+0000 7f04b9ffb640 1 -- 172.21.15.133:0/2269630473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c0005ce0 con 0x7f04c406de00 2024-02-21T23:53:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.995+0000 7f04b9ffb640 1 -- 172.21.15.133:0/2269630473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c00072d0 con 0x7f04c406de00 2024-02-21T23:53:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.995+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c414adc0 con 0x7f04c406de00 2024-02-21T23:53:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.995+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c41f3360 con 0x7f04c406de00 2024-02-21T23:53:35.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:35.997+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c41f3970 con 0x7f04c406de00 2024-02-21T23:53:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.238+0000 7f04b9ffb640 1 -- 172.21.15.133:0/2269630473 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04c001a070 con 0x7f04c406de00 2024-02-21T23:53:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.238+0000 7f04b9ffb640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f04a806b500 0x7f04a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.238+0000 7f04c8879640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f04a806b500 0x7f04a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.239+0000 7f04c8879640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f04a806b500 0x7f04a806d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04c40c6690 tx=0x7f04b4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.252+0000 7f04b9ffb640 1 -- 172.21.15.133:0/2269630473 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f04c00561d0 con 0x7f04c406de00 2024-02-21T23:53:36.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.252+0000 7f04b9ffb640 1 -- 172.21.15.133:0/2269630473 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f04c0058130 con 0x7f04c406de00 2024-02-21T23:53:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.427+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f04c406a130 con 0x7f04a806b500 2024-02-21T23:53:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.428+0000 7f04b9ffb640 1 -- 172.21.15.133:0/2269630473 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f04c406a130 con 0x7f04a806b500 2024-02-21T23:53:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 2 to deep-scrub 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f04a806b500 msgr2=0x7f04a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f04a806b500 0x7f04a806d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04c40c6690 tx=0x7f04b4009040 comp rx=0 tx=0).stop 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 msgr2=0x7f04c414ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 0x7f04c414ba70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04c000c3f0 tx=0x7f04c0001e70 comp rx=0 tx=0).stop 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 shutdown_connections 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f04a806b500 0x7f04a806d9c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c4154300 0x7f04c414c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04c40b0fe0 0x7f04c41e5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 --2- 172.21.15.133:0/2269630473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c406de00 0x7f04c414ba70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 >> 172.21.15.133:0/2269630473 conn(0x7f04c40cc620 msgr2=0x7f04c414ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 shutdown_connections 2024-02-21T23:53:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.430+0000 7f04cb2de640 1 -- 172.21.15.133:0/2269630473 wait complete. 2024-02-21T23:53:36.439 INFO:tasks.ceph:Scrubbing osd.3 2024-02-21T23:53:36.439 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-02-21T23:53:36.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.550+0000 7fc48d3dc640 1 Processor -- start 2024-02-21T23:53:36.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.550+0000 7fc48d3dc640 1 -- start start 2024-02-21T23:53:36.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.550+0000 7fc48d3dc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4881542e0 0x7fc48815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.550+0000 7fc48d3dc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc48806bfd0 0x7fc48815a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.550+0000 7fc48d3dc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48806c400 con 0x7fc4880aff30 2024-02-21T23:53:36.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.550+0000 7fc48d3dc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4880b05a0 con 0x7fc4881542e0 2024-02-21T23:53:36.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.550+0000 7fc48d3dc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48806cc40 con 0x7fc48806bfd0 2024-02-21T23:53:36.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.568+0000 7fc486ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4881542e0 0x7fc48815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.568+0000 7fc4867fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc4880aff30 0x7fc4880cdd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54038/0 (socket says 172.21.15.133:54038) 2024-02-21T23:53:36.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.568+0000 7fc4867fc640 1 -- 172.21.15.133:0/3964768136 learned_addr learned my addr 172.21.15.133:0/3964768136 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:36.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.571+0000 7fc486ffd640 1 -- 172.21.15.133:0/3964768136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc48806bfd0 msgr2=0x7fc48815a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:36.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.571+0000 7fc486ffd640 1 --2- 172.21.15.133:0/3964768136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc48806bfd0 0x7fc48815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.571+0000 7fc486ffd640 1 -- 172.21.15.133:0/3964768136 >> v1:172.21.15.133:6789/0 conn(0x7fc4880aff30 legacy=0x7fc4880cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.571+0000 7fc486ffd640 1 -- 172.21.15.133:0/3964768136 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48806c680 con 0x7fc4881542e0 2024-02-21T23:53:36.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.579+0000 7fc486ffd640 1 --2- 172.21.15.133:0/3964768136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 0x7fc48815e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc4800097a0 tx=0x7fc48002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77d1abe4edca418 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:36.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.583+0000 7fc485ffb640 1 -- 172.21.15.133:0/3964768136 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480038070 con 0x7fc4881542e0 2024-02-21T23:53:36.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.583+0000 7fc485ffb640 1 -- 172.21.15.133:0/3964768136 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc480031700 con 0x7fc4881542e0 2024-02-21T23:53:36.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.583+0000 7fc485ffb640 1 -- 172.21.15.133:0/3964768136 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480031ad0 con 0x7fc4881542e0 2024-02-21T23:53:36.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.583+0000 7fc48d3dc640 1 -- 172.21.15.133:0/3964768136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 msgr2=0x7fc48815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.583+0000 7fc48d3dc640 1 --2- 172.21.15.133:0/3964768136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 0x7fc48815e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc4800097a0 tx=0x7fc48002b7a0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.584+0000 7fc48d3dc640 1 -- 172.21.15.133:0/3964768136 shutdown_connections 2024-02-21T23:53:36.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.584+0000 7fc48d3dc640 1 --2- 172.21.15.133:0/3964768136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc48806bfd0 0x7fc48815a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.584+0000 7fc48d3dc640 1 --2- 172.21.15.133:0/3964768136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 0x7fc48815e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.584+0000 7fc48d3dc640 1 -- 172.21.15.133:0/3964768136 >> 172.21.15.133:0/3964768136 conn(0x7fc48806aec0 msgr2=0x7fc48806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:36.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.584+0000 7fc48d3dc640 1 -- 172.21.15.133:0/3964768136 shutdown_connections 2024-02-21T23:53:36.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.584+0000 7fc48d3dc640 1 -- 172.21.15.133:0/3964768136 wait complete. 2024-02-21T23:53:36.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.585+0000 7fc48d3dc640 1 Processor -- start 2024-02-21T23:53:36.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.585+0000 7fc48d3dc640 1 -- start start 2024-02-21T23:53:36.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc48d3dc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48806bfd0 0x7fc4880c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc48d3dc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4880aff30 0x7fc4881f0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc48d3dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 0x7fc4881f0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc48d3dc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48815daf0 con 0x7fc4881542e0 2024-02-21T23:53:36.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc48d3dc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48815d810 con 0x7fc48806bfd0 2024-02-21T23:53:36.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc48d3dc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48815ddd0 con 0x7fc4880aff30 2024-02-21T23:53:36.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 0x7fc4881f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48806bfd0 0x7fc4880c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc4867fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4880aff30 0x7fc4881f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48806bfd0 0x7fc4880c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58914/0 (socket says 172.21.15.133:58914) 2024-02-21T23:53:36.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc486ffd640 1 -- 172.21.15.133:0/1066873071 learned_addr learned my addr 172.21.15.133:0/1066873071 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:36.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.586+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 0x7fc4881f0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36306/0 (socket says 172.21.15.133:36306) 2024-02-21T23:53:36.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.590+0000 7fc4877fe640 1 -- 172.21.15.133:0/1066873071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4880aff30 msgr2=0x7fc4881f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.590+0000 7fc4877fe640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4880aff30 0x7fc4881f0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.590+0000 7fc4877fe640 1 -- 172.21.15.133:0/1066873071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48806bfd0 msgr2=0x7fc4880c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.590+0000 7fc4877fe640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc48806bfd0 0x7fc4880c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.590+0000 7fc4877fe640 1 -- 172.21.15.133:0/1066873071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4880af700 con 0x7fc4881542e0 2024-02-21T23:53:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.592+0000 7fc4877fe640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 0x7fc4881f0850 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc47800be50 tx=0x7fc47800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.596+0000 7fc46f7fe640 1 -- 172.21.15.133:0/1066873071 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478019070 con 0x7fc4881542e0 2024-02-21T23:53:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.596+0000 7fc46f7fe640 1 -- 172.21.15.133:0/1066873071 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc478014440 con 0x7fc4881542e0 2024-02-21T23:53:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.596+0000 7fc46f7fe640 1 -- 172.21.15.133:0/1066873071 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478014830 con 0x7fc4881542e0 2024-02-21T23:53:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.596+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4880c41d0 con 0x7fc4881542e0 2024-02-21T23:53:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.596+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4880c4790 con 0x7fc4881542e0 2024-02-21T23:53:36.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.597+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4880ca930 con 0x7fc4881542e0 2024-02-21T23:53:36.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.616+0000 7fc46f7fe640 1 -- 172.21.15.133:0/1066873071 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc47801c070 con 0x7fc4881542e0 2024-02-21T23:53:36.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.616+0000 7fc46f7fe640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc47406b520 0x7fc47406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.617+0000 7fc486ffd640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc47406b520 0x7fc47406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.617+0000 7fc486ffd640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc47406b520 0x7fc47406d9e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc480009560 tx=0x7fc480008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:36.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.621+0000 7fc46f7fe640 1 -- 172.21.15.133:0/1066873071 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fc4780570f0 con 0x7fc4881542e0 2024-02-21T23:53:36.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.621+0000 7fc46f7fe640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4740750c0 0x7fc474077560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.621+0000 7fc46f7fe640 1 -- 172.21.15.133:0/1066873071 --> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc478010e50 con 0x7fc4740750c0 2024-02-21T23:53:36.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.621+0000 7fc46f7fe640 1 -- 172.21.15.133:0/1066873071 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4780552f0 con 0x7fc4881542e0 2024-02-21T23:53:36.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.622+0000 7fc4867fc640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4740750c0 0x7fc474077560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.622+0000 7fc4867fc640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4740750c0 0x7fc474077560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:36.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.623+0000 7fc46f7fe640 1 -- 172.21.15.133:0/1066873071 <== osd.3 v2:172.21.15.133:6824/4089028447 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc478010e50 con 0x7fc4740750c0 2024-02-21T23:53:36.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 --> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc4880ce480 con 0x7fc4740750c0 2024-02-21T23:53:36.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc46f7fe640 1 -- 172.21.15.133:0/1066873071 <== osd.3 v2:172.21.15.133:6824/4089028447 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc4880ce480 con 0x7fc4740750c0 2024-02-21T23:53:36.631 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:53:36.631 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-21T23:53:36.631 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:53:36.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4740750c0 msgr2=0x7fc474077560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.133:6824/4089028447,v1:172.21.15.133:6825/4089028447] conn(0x7fc4740750c0 0x7fc474077560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc47406b520 msgr2=0x7fc47406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc47406b520 0x7fc47406d9e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc480009560 tx=0x7fc480008030 comp rx=0 tx=0).stop 2024-02-21T23:53:36.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 msgr2=0x7fc4881f0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 --2- 172.21.15.133:0/1066873071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4881542e0 0x7fc4881f0850 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc47800be50 tx=0x7fc47800bf40 comp rx=0 tx=0).stop 2024-02-21T23:53:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc4877fe640 1 -- 172.21.15.133:0/1066873071 reap_dead start 2024-02-21T23:53:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 shutdown_connections 2024-02-21T23:53:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 >> 172.21.15.133:0/1066873071 conn(0x7fc48806aec0 msgr2=0x7fc48814e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 shutdown_connections 2024-02-21T23:53:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.636+0000 7fc48d3dc640 1 -- 172.21.15.133:0/1066873071 wait complete. 2024-02-21T23:53:36.642 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-02-21T23:53:36.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.749+0000 7fcf6fa36640 1 Processor -- start 2024-02-21T23:53:36.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.750+0000 7fcf6fa36640 1 -- start start 2024-02-21T23:53:36.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.750+0000 7fcf6fa36640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcf680cdfe0 0x7fcf680c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.750+0000 7fcf6fa36640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf680ce3c0 0x7fcf68070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.750+0000 7fcf6fa36640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf680717e0 con 0x7fcf680712b0 2024-02-21T23:53:36.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.750+0000 7fcf6fa36640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf680c84b0 con 0x7fcf680cdfe0 2024-02-21T23:53:36.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.750+0000 7fcf6fa36640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf68158ca0 con 0x7fcf680ce3c0 2024-02-21T23:53:36.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.751+0000 7fcf6d7d2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcf680cdfe0 0x7fcf680c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.751+0000 7fcf6dfd3640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fcf680712b0 0x7fcf6814e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57500/0 (socket says 172.21.15.133:57500) 2024-02-21T23:53:36.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.751+0000 7fcf6dfd3640 1 -- 172.21.15.133:0/29810798 learned_addr learned my addr 172.21.15.133:0/29810798 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:36.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.751+0000 7fcf6cfd1640 1 --2- 172.21.15.133:0/29810798 >> v2:172.21.15.142:3300/0 conn(0x7fcf680ce3c0 0x7fcf68070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.753+0000 7fcf6d7d2640 1 -- 172.21.15.133:0/29810798 >> v2:172.21.15.142:3300/0 conn(0x7fcf680ce3c0 msgr2=0x7fcf68070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:36.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.753+0000 7fcf6d7d2640 1 --2- 172.21.15.133:0/29810798 >> v2:172.21.15.142:3300/0 conn(0x7fcf680ce3c0 0x7fcf68070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.754+0000 7fcf6d7d2640 1 -- 172.21.15.133:0/29810798 >> v1:172.21.15.142:6789/0 conn(0x7fcf680712b0 legacy=0x7fcf6814e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.754+0000 7fcf6d7d2640 1 -- 172.21.15.133:0/29810798 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf68158090 con 0x7fcf680cdfe0 2024-02-21T23:53:36.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.756+0000 7fcf6d7d2640 1 --2- 172.21.15.133:0/29810798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680cdfe0 0x7fcf680c3250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf680c4bb0 tx=0x7fcf6402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c54f4a4a9fac3 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:36.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.759+0000 7fcf5ffff640 1 -- 172.21.15.133:0/29810798 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64039070 con 0x7fcf680cdfe0 2024-02-21T23:53:36.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.759+0000 7fcf5ffff640 1 -- 172.21.15.133:0/29810798 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6402cc80 con 0x7fcf680cdfe0 2024-02-21T23:53:36.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.759+0000 7fcf5ffff640 1 -- 172.21.15.133:0/29810798 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64034660 con 0x7fcf680cdfe0 2024-02-21T23:53:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.760+0000 7fcf6fa36640 1 -- 172.21.15.133:0/29810798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680cdfe0 msgr2=0x7fcf680c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.760+0000 7fcf6fa36640 1 --2- 172.21.15.133:0/29810798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680cdfe0 0x7fcf680c3250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf680c4bb0 tx=0x7fcf6402bed0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.760+0000 7fcf6fa36640 1 -- 172.21.15.133:0/29810798 shutdown_connections 2024-02-21T23:53:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.760+0000 7fcf6fa36640 1 --2- 172.21.15.133:0/29810798 >> v2:172.21.15.142:3300/0 conn(0x7fcf680ce3c0 0x7fcf68070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.760+0000 7fcf6fa36640 1 --2- 172.21.15.133:0/29810798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680cdfe0 0x7fcf680c3250 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.760+0000 7fcf6fa36640 1 -- 172.21.15.133:0/29810798 >> 172.21.15.133:0/29810798 conn(0x7fcf6806aec0 msgr2=0x7fcf6806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.760+0000 7fcf6fa36640 1 -- 172.21.15.133:0/29810798 shutdown_connections 2024-02-21T23:53:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.760+0000 7fcf6fa36640 1 -- 172.21.15.133:0/29810798 wait complete. 2024-02-21T23:53:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.762+0000 7fcf6fa36640 1 Processor -- start 2024-02-21T23:53:36.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.762+0000 7fcf6fa36640 1 -- start start 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.762+0000 7fcf6fa36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf680712b0 0x7fcf6814a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.762+0000 7fcf6fa36640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf680cdfe0 0x7fcf680c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.762+0000 7fcf6fa36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680ce3c0 0x7fcf680cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.762+0000 7fcf6fa36640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68158930 con 0x7fcf680ce3c0 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.762+0000 7fcf6fa36640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68158650 con 0x7fcf680cdfe0 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.762+0000 7fcf6fa36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6814ec20 con 0x7fcf680712b0 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.763+0000 7fcf6dfd3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680ce3c0 0x7fcf680cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.763+0000 7fcf6cfd1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf680cdfe0 0x7fcf680c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.763+0000 7fcf6d7d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf680712b0 0x7fcf6814a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.763+0000 7fcf6cfd1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf680cdfe0 0x7fcf680c4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58938/0 (socket says 172.21.15.133:58938) 2024-02-21T23:53:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.763+0000 7fcf6cfd1640 1 -- 172.21.15.133:0/153819608 learned_addr learned my addr 172.21.15.133:0/153819608 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.766+0000 7fcf6dfd3640 1 -- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf680712b0 msgr2=0x7fcf6814a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.766+0000 7fcf6dfd3640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf680712b0 0x7fcf6814a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.766+0000 7fcf6dfd3640 1 -- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf680cdfe0 msgr2=0x7fcf680c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.766+0000 7fcf6dfd3640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf680cdfe0 0x7fcf680c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:36.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.766+0000 7fcf6dfd3640 1 -- 172.21.15.133:0/153819608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf680b1fc0 con 0x7fcf680ce3c0 2024-02-21T23:53:36.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.768+0000 7fcf6dfd3640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680ce3c0 0x7fcf680cbbe0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000c3f0 tx=0x7fcf6000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.772+0000 7fcf5dffb640 1 -- 172.21.15.133:0/153819608 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60017070 con 0x7fcf680ce3c0 2024-02-21T23:53:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.772+0000 7fcf5dffb640 1 -- 172.21.15.133:0/153819608 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60005ce0 con 0x7fcf680ce3c0 2024-02-21T23:53:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.772+0000 7fcf5dffb640 1 -- 172.21.15.133:0/153819608 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf600072d0 con 0x7fcf680ce3c0 2024-02-21T23:53:36.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.772+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf680b22a0 con 0x7fcf680ce3c0 2024-02-21T23:53:36.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.772+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf680cb6c0 con 0x7fcf680ce3c0 2024-02-21T23:53:36.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.773+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf680cb8c0 con 0x7fcf680ce3c0 2024-02-21T23:53:36.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.791+0000 7fcf5dffb640 1 -- 172.21.15.133:0/153819608 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf6001a070 con 0x7fcf680ce3c0 2024-02-21T23:53:36.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.791+0000 7fcf5dffb640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcf4c06b4d0 0x7fcf4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:36.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.792+0000 7fcf6d7d2640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcf4c06b4d0 0x7fcf4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:36.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:36.792+0000 7fcf6d7d2640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcf4c06b4d0 0x7fcf4c06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf64036000 tx=0x7fcf64030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:37.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.030+0000 7fcf5dffb640 1 -- 172.21.15.133:0/153819608 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fcf60056190 con 0x7fcf680ce3c0 2024-02-21T23:53:37.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.030+0000 7fcf5dffb640 1 -- 172.21.15.133:0/153819608 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fcf600580f0 con 0x7fcf680ce3c0 2024-02-21T23:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.205+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf6806a130 con 0x7fcf4c06b4d0 2024-02-21T23:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.205+0000 7fcf5dffb640 1 -- 172.21.15.133:0/153819608 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcf6806a130 con 0x7fcf4c06b4d0 2024-02-21T23:53:37.201 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 3 to deep-scrub 2024-02-21T23:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcf4c06b4d0 msgr2=0x7fcf4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcf4c06b4d0 0x7fcf4c06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf64036000 tx=0x7fcf64030000 comp rx=0 tx=0).stop 2024-02-21T23:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680ce3c0 msgr2=0x7fcf680cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680ce3c0 0x7fcf680cbbe0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000c3f0 tx=0x7fcf6000c8b0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 shutdown_connections 2024-02-21T23:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fcf4c06b4d0 0x7fcf4c06d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf680ce3c0 0x7fcf680cbbe0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf680cdfe0 0x7fcf680c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 --2- 172.21.15.133:0/153819608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf680712b0 0x7fcf6814a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 >> 172.21.15.133:0/153819608 conn(0x7fcf6806aec0 msgr2=0x7fcf6815a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:37.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 shutdown_connections 2024-02-21T23:53:37.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.208+0000 7fcf6fa36640 1 -- 172.21.15.133:0/153819608 wait complete. 2024-02-21T23:53:37.214 INFO:tasks.ceph:Scrubbing osd.4 2024-02-21T23:53:37.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-02-21T23:53:37.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.324+0000 7efd6d4d5640 1 Processor -- start 2024-02-21T23:53:37.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.324+0000 7efd6d4d5640 1 -- start start 2024-02-21T23:53:37.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.325+0000 7efd6d4d5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd68071f60 0x7efd680c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.325+0000 7efd6d4d5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efd680c7ac0 0x7efd680cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.325+0000 7efd6d4d5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd680c53e0 con 0x7efd68150a30 2024-02-21T23:53:37.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.325+0000 7efd6d4d5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd68071860 con 0x7efd68071f60 2024-02-21T23:53:37.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.325+0000 7efd6d4d5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd68072540 con 0x7efd680c7ac0 2024-02-21T23:53:37.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.325+0000 7efd667fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd68071f60 0x7efd680c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.326+0000 7efd677fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efd680c7ac0 0x7efd680cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.326+0000 7efd677fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efd680c7ac0 0x7efd680cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58940/0 (socket says 172.21.15.133:58940) 2024-02-21T23:53:37.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.326+0000 7efd677fe640 1 -- 172.21.15.133:0/2652695796 learned_addr learned my addr 172.21.15.133:0/2652695796 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:37.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.328+0000 7efd677fe640 1 -- 172.21.15.133:0/2652695796 >> v2:172.21.15.133:3300/0 conn(0x7efd68071f60 msgr2=0x7efd680c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:37.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.328+0000 7efd677fe640 1 --2- 172.21.15.133:0/2652695796 >> v2:172.21.15.133:3300/0 conn(0x7efd68071f60 0x7efd680c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.328+0000 7efd677fe640 1 -- 172.21.15.133:0/2652695796 >> v1:172.21.15.133:6789/0 conn(0x7efd68150a30 legacy=0x7efd68071150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.328+0000 7efd677fe640 1 -- 172.21.15.133:0/2652695796 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd680c5680 con 0x7efd680c7ac0 2024-02-21T23:53:37.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.330+0000 7efd677fe640 1 --2- 172.21.15.133:0/2652695796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd680c7ac0 0x7efd680cb660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd680c2300 tx=0x7efd6002ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2b6dff7bdb623b server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:37.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.334+0000 7efd65ffb640 1 -- 172.21.15.133:0/2652695796 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6002f850 con 0x7efd680c7ac0 2024-02-21T23:53:37.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.334+0000 7efd65ffb640 1 -- 172.21.15.133:0/2652695796 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6002f9f0 con 0x7efd680c7ac0 2024-02-21T23:53:37.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.334+0000 7efd65ffb640 1 -- 172.21.15.133:0/2652695796 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6002fdc0 con 0x7efd680c7ac0 2024-02-21T23:53:37.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.334+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2652695796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd680c7ac0 msgr2=0x7efd680cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.334+0000 7efd6d4d5640 1 --2- 172.21.15.133:0/2652695796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd680c7ac0 0x7efd680cb660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd680c2300 tx=0x7efd6002ba30 comp rx=0 tx=0).stop 2024-02-21T23:53:37.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.334+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2652695796 shutdown_connections 2024-02-21T23:53:37.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.334+0000 7efd6d4d5640 1 --2- 172.21.15.133:0/2652695796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd680c7ac0 0x7efd680cb660 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.334+0000 7efd6d4d5640 1 --2- 172.21.15.133:0/2652695796 >> v2:172.21.15.133:3300/0 conn(0x7efd68071f60 0x7efd680c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.334+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2652695796 >> 172.21.15.133:0/2652695796 conn(0x7efd6806aec0 msgr2=0x7efd6806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:37.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.335+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2652695796 shutdown_connections 2024-02-21T23:53:37.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.335+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2652695796 wait complete. 2024-02-21T23:53:37.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.336+0000 7efd6d4d5640 1 Processor -- start 2024-02-21T23:53:37.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.336+0000 7efd6d4d5640 1 -- start start 2024-02-21T23:53:37.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd6d4d5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68071f60 0x7efd68142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd6d4d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd680c7ac0 0x7efd68138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd6d4d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd68150a30 0x7efd6813eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd6d4d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd68072a70 con 0x7efd68150a30 2024-02-21T23:53:37.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd6d4d5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd68072790 con 0x7efd68071f60 2024-02-21T23:53:37.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd6d4d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd680cbba0 con 0x7efd680c7ac0 2024-02-21T23:53:37.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd677fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd68150a30 0x7efd6813eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd66ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68071f60 0x7efd68142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd667fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd680c7ac0 0x7efd68138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd66ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68071f60 0x7efd68142f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58950/0 (socket says 172.21.15.133:58950) 2024-02-21T23:53:37.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.337+0000 7efd66ffd640 1 -- 172.21.15.133:0/2385380795 learned_addr learned my addr 172.21.15.133:0/2385380795 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:37.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.341+0000 7efd677fe640 1 -- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd680c7ac0 msgr2=0x7efd68138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.341+0000 7efd677fe640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd680c7ac0 0x7efd68138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.341+0000 7efd677fe640 1 -- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68071f60 msgr2=0x7efd68142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.341+0000 7efd677fe640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68071f60 0x7efd68142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.341+0000 7efd677fe640 1 -- 172.21.15.133:0/2385380795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68139850 con 0x7efd68150a30 2024-02-21T23:53:37.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.343+0000 7efd677fe640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd68150a30 0x7efd6813eff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd60033000 tx=0x7efd60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:37.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.346+0000 7efd4f7fe640 1 -- 172.21.15.133:0/2385380795 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60037c00 con 0x7efd68150a30 2024-02-21T23:53:37.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.346+0000 7efd4f7fe640 1 -- 172.21.15.133:0/2385380795 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd60046070 con 0x7efd68150a30 2024-02-21T23:53:37.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.346+0000 7efd4f7fe640 1 -- 172.21.15.133:0/2385380795 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60041400 con 0x7efd68150a30 2024-02-21T23:53:37.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.346+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd68139af0 con 0x7efd68150a30 2024-02-21T23:53:37.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.346+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd68138160 con 0x7efd68150a30 2024-02-21T23:53:37.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.347+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd68139db0 con 0x7efd68150a30 2024-02-21T23:53:37.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.367+0000 7efd4f7fe640 1 -- 172.21.15.133:0/2385380795 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd60040070 con 0x7efd68150a30 2024-02-21T23:53:37.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.367+0000 7efd4f7fe640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd5406b550 0x7efd5406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.367+0000 7efd66ffd640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd5406b550 0x7efd5406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.368+0000 7efd66ffd640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd5406b550 0x7efd5406da10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd58007fd0 tx=0x7efd58009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.371+0000 7efd4f7fe640 1 -- 172.21.15.133:0/2385380795 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7efd60079980 con 0x7efd68150a30 2024-02-21T23:53:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.371+0000 7efd4f7fe640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7efd540750e0 0x7efd54077580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.371+0000 7efd4f7fe640 1 -- 172.21.15.133:0/2385380795 --> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd60006810 con 0x7efd540750e0 2024-02-21T23:53:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.371+0000 7efd4f7fe640 1 -- 172.21.15.133:0/2385380795 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd60077b80 con 0x7efd68150a30 2024-02-21T23:53:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.371+0000 7efd667fc640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7efd540750e0 0x7efd54077580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.371+0000 7efd667fc640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7efd540750e0 0x7efd54077580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:37.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.372+0000 7efd4f7fe640 1 -- 172.21.15.133:0/2385380795 <== osd.4 v2:172.21.15.142:6826/2259485122 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efd60006810 con 0x7efd540750e0 2024-02-21T23:53:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 --> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efd6806a130 con 0x7efd540750e0 2024-02-21T23:53:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd4f7fe640 1 -- 172.21.15.133:0/2385380795 <== osd.4 v2:172.21.15.142:6826/2259485122 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7efd6806a130 con 0x7efd540750e0 2024-02-21T23:53:37.381 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:53:37.381 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-21T23:53:37.381 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:53:37.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7efd540750e0 msgr2=0x7efd54077580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6826/2259485122,v1:172.21.15.142:6827/2259485122] conn(0x7efd540750e0 0x7efd54077580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd5406b550 msgr2=0x7efd5406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7efd5406b550 0x7efd5406da10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd58007fd0 tx=0x7efd58009040 comp rx=0 tx=0).stop 2024-02-21T23:53:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd68150a30 msgr2=0x7efd6813eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 --2- 172.21.15.133:0/2385380795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd68150a30 0x7efd6813eff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd60033000 tx=0x7efd60004730 comp rx=0 tx=0).stop 2024-02-21T23:53:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd677fe640 1 -- 172.21.15.133:0/2385380795 reap_dead start 2024-02-21T23:53:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 shutdown_connections 2024-02-21T23:53:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 >> 172.21.15.133:0/2385380795 conn(0x7efd6806aec0 msgr2=0x7efd6806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 shutdown_connections 2024-02-21T23:53:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.386+0000 7efd6d4d5640 1 -- 172.21.15.133:0/2385380795 wait complete. 2024-02-21T23:53:37.393 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-02-21T23:53:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.500+0000 7f50fa974640 1 Processor -- start 2024-02-21T23:53:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.500+0000 7f50fa974640 1 -- start start 2024-02-21T23:53:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.501+0000 7f50fa974640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f4071f60 0x7f50f40c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.501+0000 7f50fa974640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f50f40c7ac0 0x7f50f406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.501+0000 7f50fa974640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f40c53e0 con 0x7f50f4150a30 2024-02-21T23:53:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.501+0000 7f50fa974640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f4071860 con 0x7f50f40c7ac0 2024-02-21T23:53:37.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.501+0000 7f50fa974640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4072540 con 0x7f50f4071f60 2024-02-21T23:53:37.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.502+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f4071f60 0x7f50f40c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.502+0000 7f50f8f11640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f50f40c7ac0 0x7f50f406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.502+0000 7f50f3fff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f50f4150a30 0x7f50f4071150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57514/0 (socket says 172.21.15.133:57514) 2024-02-21T23:53:37.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.502+0000 7f50f3fff640 1 -- 172.21.15.133:0/531259390 learned_addr learned my addr 172.21.15.133:0/531259390 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.505+0000 7f50f37fe640 1 -- 172.21.15.133:0/531259390 >> v2:172.21.15.142:3300/0 conn(0x7f50f40c7ac0 msgr2=0x7f50f406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.505+0000 7f50f37fe640 1 --2- 172.21.15.133:0/531259390 >> v2:172.21.15.142:3300/0 conn(0x7f50f40c7ac0 0x7f50f406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.505+0000 7f50f37fe640 1 -- 172.21.15.133:0/531259390 >> v1:172.21.15.142:6789/0 conn(0x7f50f4150a30 legacy=0x7f50f4071150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.505+0000 7f50f37fe640 1 -- 172.21.15.133:0/531259390 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f40c5680 con 0x7f50f4071f60 2024-02-21T23:53:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.507+0000 7f50f37fe640 1 --2- 172.21.15.133:0/531259390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f4071f60 0x7f50f40c2b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f50f40726c0 tx=0x7f50e802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e198aca65697737 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:37.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.510+0000 7f50f2ffd640 1 -- 172.21.15.133:0/531259390 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8035020 con 0x7f50f4071f60 2024-02-21T23:53:37.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.510+0000 7f50f2ffd640 1 -- 172.21.15.133:0/531259390 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e8031890 con 0x7f50f4071f60 2024-02-21T23:53:37.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.510+0000 7f50f2ffd640 1 -- 172.21.15.133:0/531259390 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8031c60 con 0x7f50f4071f60 2024-02-21T23:53:37.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.511+0000 7f50fa974640 1 -- 172.21.15.133:0/531259390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f4071f60 msgr2=0x7f50f40c2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.511+0000 7f50fa974640 1 --2- 172.21.15.133:0/531259390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f4071f60 0x7f50f40c2b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f50f40726c0 tx=0x7f50e802b930 comp rx=0 tx=0).stop 2024-02-21T23:53:37.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.511+0000 7f50fa974640 1 -- 172.21.15.133:0/531259390 shutdown_connections 2024-02-21T23:53:37.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.511+0000 7f50fa974640 1 --2- 172.21.15.133:0/531259390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f4071f60 0x7f50f40c2b40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.511+0000 7f50fa974640 1 --2- 172.21.15.133:0/531259390 >> v2:172.21.15.142:3300/0 conn(0x7f50f40c7ac0 0x7f50f406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.511+0000 7f50fa974640 1 -- 172.21.15.133:0/531259390 >> 172.21.15.133:0/531259390 conn(0x7f50f406aec0 msgr2=0x7f50f406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:37.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.511+0000 7f50fa974640 1 -- 172.21.15.133:0/531259390 shutdown_connections 2024-02-21T23:53:37.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.511+0000 7f50fa974640 1 -- 172.21.15.133:0/531259390 wait complete. 2024-02-21T23:53:37.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.512+0000 7f50fa974640 1 Processor -- start 2024-02-21T23:53:37.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.512+0000 7f50fa974640 1 -- start start 2024-02-21T23:53:37.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.513+0000 7f50fa974640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50f4071f60 0x7f50f41460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.513+0000 7f50fa974640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f40c7ac0 0x7f50f414ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.513+0000 7f50fa974640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f4150a30 0x7f50f414f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.513+0000 7f50fa974640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4072a70 con 0x7f50f4071f60 2024-02-21T23:53:37.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.513+0000 7f50fa974640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4072790 con 0x7f50f4150a30 2024-02-21T23:53:37.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.513+0000 7f50fa974640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4150e10 con 0x7f50f40c7ac0 2024-02-21T23:53:37.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.514+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f40c7ac0 0x7f50f414ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.514+0000 7f50f8f11640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f4150a30 0x7f50f414f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.514+0000 7f50f8f11640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f4150a30 0x7f50f414f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58974/0 (socket says 172.21.15.133:58974) 2024-02-21T23:53:37.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.514+0000 7f50f8f11640 1 -- 172.21.15.133:0/1657986665 learned_addr learned my addr 172.21.15.133:0/1657986665 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:37.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.515+0000 7f50f3fff640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50f4071f60 0x7f50f41460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.517+0000 7f50f37fe640 1 -- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f4150a30 msgr2=0x7f50f414f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.517+0000 7f50f37fe640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f4150a30 0x7f50f414f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.517+0000 7f50f37fe640 1 -- 172.21.15.133:0/1657986665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50f4071f60 msgr2=0x7f50f41460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.517+0000 7f50f37fe640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50f4071f60 0x7f50f41460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.517+0000 7f50f37fe640 1 -- 172.21.15.133:0/1657986665 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f41eef70 con 0x7f50f40c7ac0 2024-02-21T23:53:37.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.522+0000 7f50f37fe640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f40c7ac0 0x7f50f414ed00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f50f406bea0 tx=0x7f50e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.527+0000 7f50f0ff9640 1 -- 172.21.15.133:0/1657986665 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8035040 con 0x7f50f40c7ac0 2024-02-21T23:53:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.527+0000 7f50f0ff9640 1 -- 172.21.15.133:0/1657986665 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e803f070 con 0x7f50f40c7ac0 2024-02-21T23:53:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.527+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f41ee3f0 con 0x7f50f40c7ac0 2024-02-21T23:53:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.527+0000 7f50f0ff9640 1 -- 172.21.15.133:0/1657986665 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8046670 con 0x7f50f40c7ac0 2024-02-21T23:53:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.527+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f41ee9b0 con 0x7f50f40c7ac0 2024-02-21T23:53:37.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.528+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f40c3d10 con 0x7f50f40c7ac0 2024-02-21T23:53:37.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.798+0000 7f50f0ff9640 1 -- 172.21.15.133:0/1657986665 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50e8038020 con 0x7f50f40c7ac0 2024-02-21T23:53:37.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.799+0000 7f50f0ff9640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f50e006b4d0 0x7f50e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:37.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.799+0000 7f50f3fff640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f50e006b4d0 0x7f50e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:37.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.799+0000 7f50f3fff640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f50e006b4d0 0x7f50e006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f50e4000f60 tx=0x7f50e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:37.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.806+0000 7f50f0ff9640 1 -- 172.21.15.133:0/1657986665 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f50e8078f80 con 0x7f50f40c7ac0 2024-02-21T23:53:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.816+0000 7f50f0ff9640 1 -- 172.21.15.133:0/1657986665 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f50f40c3d10 con 0x7f50f40c7ac0 2024-02-21T23:53:37.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.994+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f50f406a130 con 0x7f50e006b4d0 2024-02-21T23:53:37.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.994+0000 7f50f0ff9640 1 -- 172.21.15.133:0/1657986665 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f50f406a130 con 0x7f50e006b4d0 2024-02-21T23:53:37.989 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 4 to deep-scrub 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f50e006b4d0 msgr2=0x7f50e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f50e006b4d0 0x7f50e006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f50e4000f60 tx=0x7f50e4009040 comp rx=0 tx=0).stop 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f40c7ac0 msgr2=0x7f50f414ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f40c7ac0 0x7f50f414ed00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f50f406bea0 tx=0x7f50e80046c0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 shutdown_connections 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f50e006b4d0 0x7f50e006d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f4150a30 0x7f50f414f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50f40c7ac0 0x7f50f414ed00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 --2- 172.21.15.133:0/1657986665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50f4071f60 0x7f50f41460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 >> 172.21.15.133:0/1657986665 conn(0x7f50f406aec0 msgr2=0x7f50f40c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:37.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 shutdown_connections 2024-02-21T23:53:37.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:37.996+0000 7f50fa974640 1 -- 172.21.15.133:0/1657986665 wait complete. 2024-02-21T23:53:38.003 INFO:tasks.ceph:Scrubbing osd.5 2024-02-21T23:53:38.003 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-02-21T23:53:38.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.113+0000 7ff01355e640 1 Processor -- start 2024-02-21T23:53:38.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.113+0000 7ff01355e640 1 -- start start 2024-02-21T23:53:38.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.114+0000 7ff01355e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff00c0c9450 0x7ff00c072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.114+0000 7ff01355e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00c0c7790 0x7ff00c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.114+0000 7ff01355e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00c072fa0 con 0x7ff00c0c9830 2024-02-21T23:53:38.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.114+0000 7ff01355e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00c073210 con 0x7ff00c0c9450 2024-02-21T23:53:38.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.114+0000 7ff01355e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c15a280 con 0x7ff00c0c7790 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.115+0000 7ff010af9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00c0c7790 0x7ff00c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.115+0000 7ff0112fa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff00c0c9450 0x7ff00c072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.115+0000 7ff011afb640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7ff00c0c9830 0x7ff00c150940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57522/0 (socket says 172.21.15.133:57522) 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.115+0000 7ff011afb640 1 -- 172.21.15.133:0/2447195658 learned_addr learned my addr 172.21.15.133:0/2447195658 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.118+0000 7ff0112fa640 1 -- 172.21.15.133:0/2447195658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00c0c7790 msgr2=0x7ff00c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.118+0000 7ff0112fa640 1 --2- 172.21.15.133:0/2447195658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00c0c7790 0x7ff00c15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.118+0000 7ff0112fa640 1 -- 172.21.15.133:0/2447195658 >> v1:172.21.15.142:6789/0 conn(0x7ff00c0c9830 legacy=0x7ff00c150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.118+0000 7ff0112fa640 1 -- 172.21.15.133:0/2447195658 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c15a560 con 0x7ff00c0c9450 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.122+0000 7ff0112fa640 1 --2- 172.21.15.133:0/2447195658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9450 0x7ff00c072a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff008009d90 tx=0x7ff00802bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701a7765f2c12f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.126+0000 7ff003fff640 1 -- 172.21.15.133:0/2447195658 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008039070 con 0x7ff00c0c9450 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.126+0000 7ff003fff640 1 -- 172.21.15.133:0/2447195658 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00802cce0 con 0x7ff00c0c9450 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.126+0000 7ff003fff640 1 -- 172.21.15.133:0/2447195658 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008034660 con 0x7ff00c0c9450 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.126+0000 7ff01355e640 1 -- 172.21.15.133:0/2447195658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9450 msgr2=0x7ff00c072a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.126+0000 7ff01355e640 1 --2- 172.21.15.133:0/2447195658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9450 0x7ff00c072a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff008009d90 tx=0x7ff00802bf30 comp rx=0 tx=0).stop 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.126+0000 7ff01355e640 1 -- 172.21.15.133:0/2447195658 shutdown_connections 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.126+0000 7ff01355e640 1 --2- 172.21.15.133:0/2447195658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00c0c7790 0x7ff00c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.126+0000 7ff01355e640 1 --2- 172.21.15.133:0/2447195658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9450 0x7ff00c072a60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.126+0000 7ff01355e640 1 -- 172.21.15.133:0/2447195658 >> 172.21.15.133:0/2447195658 conn(0x7ff00c06bfd0 msgr2=0x7ff00c06c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.127+0000 7ff01355e640 1 -- 172.21.15.133:0/2447195658 shutdown_connections 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.127+0000 7ff01355e640 1 -- 172.21.15.133:0/2447195658 wait complete. 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.128+0000 7ff01355e640 1 Processor -- start 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.128+0000 7ff01355e640 1 -- start start 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.128+0000 7ff01355e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00c0c7790 0x7ff00c0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.128+0000 7ff01355e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff00c0c9450 0x7ff00c14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.128+0000 7ff01355e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9830 0x7ff00c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.128+0000 7ff01355e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c151050 con 0x7ff00c0c9450 2024-02-21T23:53:38.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.128+0000 7ff01355e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c15ab20 con 0x7ff00c0c9830 2024-02-21T23:53:38.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.128+0000 7ff01355e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c1511d0 con 0x7ff00c0c7790 2024-02-21T23:53:38.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.129+0000 7ff011afb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9830 0x7ff00c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.129+0000 7ff0112fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00c0c7790 0x7ff00c0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.129+0000 7ff010af9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff00c0c9450 0x7ff00c14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.129+0000 7ff011afb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9830 0x7ff00c15d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58980/0 (socket says 172.21.15.133:58980) 2024-02-21T23:53:38.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.129+0000 7ff011afb640 1 -- 172.21.15.133:0/126490727 learned_addr learned my addr 172.21.15.133:0/126490727 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:38.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.132+0000 7ff011afb640 1 -- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00c0c7790 msgr2=0x7ff00c0cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.132+0000 7ff011afb640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00c0c7790 0x7ff00c0cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.132+0000 7ff011afb640 1 -- 172.21.15.133:0/126490727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff00c0c9450 msgr2=0x7ff00c14ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.132+0000 7ff011afb640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff00c0c9450 0x7ff00c14ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.132+0000 7ff011afb640 1 -- 172.21.15.133:0/126490727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c0c5280 con 0x7ff00c0c9830 2024-02-21T23:53:38.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.134+0000 7ff011afb640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9830 0x7ff00c15d770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c390 tx=0x7ff00400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.138+0000 7ff001ffb640 1 -- 172.21.15.133:0/126490727 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004017070 con 0x7ff00c0c9830 2024-02-21T23:53:38.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.138+0000 7ff001ffb640 1 -- 172.21.15.133:0/126490727 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff004005ce0 con 0x7ff00c0c9830 2024-02-21T23:53:38.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.138+0000 7ff001ffb640 1 -- 172.21.15.133:0/126490727 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0040072d0 con 0x7ff00c0c9830 2024-02-21T23:53:38.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.138+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00c0c5560 con 0x7ff00c0c9830 2024-02-21T23:53:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.138+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00c0c5b20 con 0x7ff00c0c9830 2024-02-21T23:53:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.139+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff00c15d350 con 0x7ff00c0c9830 2024-02-21T23:53:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.158+0000 7ff001ffb640 1 -- 172.21.15.133:0/126490727 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff00401a070 con 0x7ff00c0c9830 2024-02-21T23:53:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.158+0000 7ff001ffb640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feff006b520 0x7feff006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.159+0000 7ff0112fa640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feff006b520 0x7feff006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.159+0000 7ff0112fa640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feff006b520 0x7feff006d9e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff008036000 tx=0x7ff008030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.161+0000 7ff001ffb640 1 -- 172.21.15.133:0/126490727 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7ff004000f60 con 0x7ff00c0c9830 2024-02-21T23:53:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.161+0000 7ff001ffb640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7feff0075040 0x7feff00774e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.161+0000 7ff001ffb640 1 -- 172.21.15.133:0/126490727 --> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff00401de80 con 0x7feff0075040 2024-02-21T23:53:38.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.161+0000 7ff010af9640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7feff0075040 0x7feff00774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.162+0000 7ff010af9640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7feff0075040 0x7feff00774e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.162+0000 7ff001ffb640 1 -- 172.21.15.133:0/126490727 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff004054600 con 0x7ff00c0c9830 2024-02-21T23:53:38.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.163+0000 7ff001ffb640 1 -- 172.21.15.133:0/126490727 <== osd.5 v2:172.21.15.142:6818/3321674168 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff00401de80 con 0x7feff0075040 2024-02-21T23:53:38.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.175+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 --> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff00c06a130 con 0x7feff0075040 2024-02-21T23:53:38.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.175+0000 7ff001ffb640 1 -- 172.21.15.133:0/126490727 <== osd.5 v2:172.21.15.142:6818/3321674168 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff00c06a130 con 0x7feff0075040 2024-02-21T23:53:38.189 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:53:38.189 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-21T23:53:38.189 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:53:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7feff0075040 msgr2=0x7feff00774e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6818/3321674168,v1:172.21.15.142:6819/3321674168] conn(0x7feff0075040 0x7feff00774e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feff006b520 msgr2=0x7feff006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7feff006b520 0x7feff006d9e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff008036000 tx=0x7ff008030000 comp rx=0 tx=0).stop 2024-02-21T23:53:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9830 msgr2=0x7ff00c15d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 --2- 172.21.15.133:0/126490727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff00c0c9830 0x7ff00c15d770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c390 tx=0x7ff00400c850 comp rx=0 tx=0).stop 2024-02-21T23:53:38.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff011afb640 1 -- 172.21.15.133:0/126490727 reap_dead start 2024-02-21T23:53:38.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 shutdown_connections 2024-02-21T23:53:38.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 >> 172.21.15.133:0/126490727 conn(0x7ff00c06bfd0 msgr2=0x7ff00c0c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:38.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 shutdown_connections 2024-02-21T23:53:38.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.176+0000 7ff01355e640 1 -- 172.21.15.133:0/126490727 wait complete. 2024-02-21T23:53:38.193 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-02-21T23:53:38.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.301+0000 7fe75a6b7640 1 Processor -- start 2024-02-21T23:53:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.301+0000 7fe75a6b7640 1 -- start start 2024-02-21T23:53:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.302+0000 7fe75a6b7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe754071950 0x7fe754072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.302+0000 7fe75a6b7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7540c7370 con 0x7fe7541584b0 2024-02-21T23:53:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.302+0000 7fe75a6b7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7540c7eb0 con 0x7fe754072230 2024-02-21T23:53:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.302+0000 7fe75a6b7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe754152640 con 0x7fe754071950 2024-02-21T23:53:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.303+0000 7fe753fff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fe754072230 0x7fe7540c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57534/0 (socket says 172.21.15.133:57534) 2024-02-21T23:53:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.303+0000 7fe753fff640 1 -- 172.21.15.133:0/994808139 learned_addr learned my addr 172.21.15.133:0/994808139 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.303+0000 7fe7537fe640 1 --2- 172.21.15.133:0/994808139 >> v2:172.21.15.133:3300/0 conn(0x7fe754071950 0x7fe754072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.305+0000 7fe752ffd640 1 -- 172.21.15.133:0/994808139 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506966249 0 0) 0x7fe7540c7eb0 con 0x7fe754072230 2024-02-21T23:53:38.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.305+0000 7fe752ffd640 1 -- 172.21.15.133:0/994808139 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe73c003520 con 0x7fe754072230 2024-02-21T23:53:38.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.306+0000 7fe7537fe640 1 -- 172.21.15.133:0/994808139 >> v1:172.21.15.142:6789/0 conn(0x7fe754072230 legacy=0x7fe7540c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.306+0000 7fe7537fe640 1 -- 172.21.15.133:0/994808139 >> v1:172.21.15.133:6789/0 conn(0x7fe7541584b0 legacy=0x7fe754162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.306+0000 7fe7537fe640 1 -- 172.21.15.133:0/994808139 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7540c7610 con 0x7fe754071950 2024-02-21T23:53:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.311+0000 7fe7537fe640 1 --2- 172.21.15.133:0/994808139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 0x7fe754072610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe7440097a0 tx=0x7fe74402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67525c9883327fcf server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.314+0000 7fe752ffd640 1 -- 172.21.15.133:0/994808139 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744034070 con 0x7fe754071950 2024-02-21T23:53:38.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.314+0000 7fe752ffd640 1 -- 172.21.15.133:0/994808139 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe744030610 con 0x7fe754071950 2024-02-21T23:53:38.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.314+0000 7fe752ffd640 1 -- 172.21.15.133:0/994808139 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7440309e0 con 0x7fe754071950 2024-02-21T23:53:38.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.315+0000 7fe75a6b7640 1 -- 172.21.15.133:0/994808139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 msgr2=0x7fe754072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.315+0000 7fe75a6b7640 1 --2- 172.21.15.133:0/994808139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 0x7fe754072610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe7440097a0 tx=0x7fe74402b960 comp rx=0 tx=0).stop 2024-02-21T23:53:38.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.315+0000 7fe75a6b7640 1 -- 172.21.15.133:0/994808139 shutdown_connections 2024-02-21T23:53:38.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.315+0000 7fe75a6b7640 1 --2- 172.21.15.133:0/994808139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 0x7fe754072610 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.315+0000 7fe75a6b7640 1 -- 172.21.15.133:0/994808139 >> 172.21.15.133:0/994808139 conn(0x7fe75406f6e0 msgr2=0x7fe75406faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:38.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.315+0000 7fe75a6b7640 1 -- 172.21.15.133:0/994808139 shutdown_connections 2024-02-21T23:53:38.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.315+0000 7fe75a6b7640 1 -- 172.21.15.133:0/994808139 wait complete. 2024-02-21T23:53:38.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.317+0000 7fe75a6b7640 1 Processor -- start 2024-02-21T23:53:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.317+0000 7fe75a6b7640 1 -- start start 2024-02-21T23:53:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.317+0000 7fe75a6b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 0x7fe754156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.317+0000 7fe75a6b7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe754072230 0x7fe7541ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.317+0000 7fe75a6b7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7541584b0 0x7fe7541445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.317+0000 7fe75a6b7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540c7bd0 con 0x7fe754071950 2024-02-21T23:53:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.317+0000 7fe75a6b7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe754158020 con 0x7fe754072230 2024-02-21T23:53:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.317+0000 7fe75a6b7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540c6f40 con 0x7fe7541584b0 2024-02-21T23:53:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.318+0000 7fe753fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 0x7fe754156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.318+0000 7fe7537fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe754072230 0x7fe7541ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.318+0000 7fe758c54640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7541584b0 0x7fe7541445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.318+0000 7fe7537fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe754072230 0x7fe7541ee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:58982/0 (socket says 172.21.15.133:58982) 2024-02-21T23:53:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.318+0000 7fe7537fe640 1 -- 172.21.15.133:0/4090176319 learned_addr learned my addr 172.21.15.133:0/4090176319 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.322+0000 7fe753fff640 1 -- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7541584b0 msgr2=0x7fe7541445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.322+0000 7fe753fff640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7541584b0 0x7fe7541445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.322+0000 7fe753fff640 1 -- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe754072230 msgr2=0x7fe7541ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.322+0000 7fe753fff640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe754072230 0x7fe7541ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.322+0000 7fe753fff640 1 -- 172.21.15.133:0/4090176319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754159be0 con 0x7fe754071950 2024-02-21T23:53:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.322+0000 7fe7537fe640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe754072230 0x7fe7541ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:53:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.324+0000 7fe753fff640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 0x7fe754156a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe74800c400 tx=0x7fe74800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.327+0000 7fe750ff9640 1 -- 172.21.15.133:0/4090176319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe748017070 con 0x7fe754071950 2024-02-21T23:53:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.327+0000 7fe750ff9640 1 -- 172.21.15.133:0/4090176319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe748005ce0 con 0x7fe754071950 2024-02-21T23:53:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.328+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7541f18a0 con 0x7fe754071950 2024-02-21T23:53:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.328+0000 7fe750ff9640 1 -- 172.21.15.133:0/4090176319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7480072f0 con 0x7fe754071950 2024-02-21T23:53:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.328+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7541f1b60 con 0x7fe754071950 2024-02-21T23:53:38.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.329+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75406a130 con 0x7fe754071950 2024-02-21T23:53:38.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.349+0000 7fe750ff9640 1 -- 172.21.15.133:0/4090176319 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe74801a070 con 0x7fe754071950 2024-02-21T23:53:38.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.349+0000 7fe750ff9640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe74006b520 0x7fe74006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.350+0000 7fe7537fe640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe74006b520 0x7fe74006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.350+0000 7fe7537fe640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe74006b520 0x7fe74006d9e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe74402b9a0 tx=0x7fe74403b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.589+0000 7fe750ff9640 1 -- 172.21.15.133:0/4090176319 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fe7480561f0 con 0x7fe754071950 2024-02-21T23:53:38.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.589+0000 7fe750ff9640 1 -- 172.21.15.133:0/4090176319 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fe748021020 con 0x7fe754071950 2024-02-21T23:53:38.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.767+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe754151830 con 0x7fe74006b520 2024-02-21T23:53:38.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.767+0000 7fe750ff9640 1 -- 172.21.15.133:0/4090176319 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe754151830 con 0x7fe74006b520 2024-02-21T23:53:38.763 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 5 to deep-scrub 2024-02-21T23:53:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe74006b520 msgr2=0x7fe74006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe74006b520 0x7fe74006d9e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe74402b9a0 tx=0x7fe74403b040 comp rx=0 tx=0).stop 2024-02-21T23:53:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 msgr2=0x7fe754156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 0x7fe754156a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe74800c400 tx=0x7fe74800c8c0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 shutdown_connections 2024-02-21T23:53:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe74006b520 0x7fe74006d9e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7541584b0 0x7fe7541445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe754072230 0x7fe7541ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 --2- 172.21.15.133:0/4090176319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe754071950 0x7fe754156a00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 >> 172.21.15.133:0/4090176319 conn(0x7fe75406f6e0 msgr2=0x7fe75406e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 shutdown_connections 2024-02-21T23:53:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.770+0000 7fe75a6b7640 1 -- 172.21.15.133:0/4090176319 wait complete. 2024-02-21T23:53:38.776 INFO:tasks.ceph:Scrubbing osd.6 2024-02-21T23:53:38.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-02-21T23:53:38.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.886+0000 7fb63946d640 1 Processor -- start 2024-02-21T23:53:38.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.887+0000 7fb63946d640 1 -- start start 2024-02-21T23:53:38.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.887+0000 7fb63946d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb634071f60 0x7fb6340c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.887+0000 7fb63946d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6340c7ac0 0x7fb63406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.887+0000 7fb63946d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340c53e0 con 0x7fb634150a30 2024-02-21T23:53:38.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.887+0000 7fb63946d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb634071860 con 0x7fb634071f60 2024-02-21T23:53:38.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.887+0000 7fb63946d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb634072540 con 0x7fb6340c7ac0 2024-02-21T23:53:38.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.888+0000 7fb6337fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6340c7ac0 0x7fb63406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.888+0000 7fb632ffd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fb634150a30 0x7fb634071150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57550/0 (socket says 172.21.15.133:57550) 2024-02-21T23:53:38.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.888+0000 7fb632ffd640 1 -- 172.21.15.133:0/3893211408 learned_addr learned my addr 172.21.15.133:0/3893211408 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:38.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.888+0000 7fb6327fc640 1 --2- 172.21.15.133:0/3893211408 >> v2:172.21.15.142:3300/0 conn(0x7fb634071f60 0x7fb6340c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.891+0000 7fb6337fe640 1 -- 172.21.15.133:0/3893211408 >> v2:172.21.15.142:3300/0 conn(0x7fb634071f60 msgr2=0x7fb6340c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:38.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.891+0000 7fb6337fe640 1 --2- 172.21.15.133:0/3893211408 >> v2:172.21.15.142:3300/0 conn(0x7fb634071f60 0x7fb6340c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.891+0000 7fb6337fe640 1 -- 172.21.15.133:0/3893211408 >> v1:172.21.15.142:6789/0 conn(0x7fb634150a30 legacy=0x7fb634071150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.891+0000 7fb6337fe640 1 -- 172.21.15.133:0/3893211408 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340c5680 con 0x7fb6340c7ac0 2024-02-21T23:53:38.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.894+0000 7fb6337fe640 1 --2- 172.21.15.133:0/3893211408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6340c7ac0 0x7fb63406e8b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb6340726c0 tx=0x7fb62c02d030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ed1d691697e694 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.898+0000 7fb631ffb640 1 -- 172.21.15.133:0/3893211408 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c03a070 con 0x7fb6340c7ac0 2024-02-21T23:53:38.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.898+0000 7fb631ffb640 1 -- 172.21.15.133:0/3893211408 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c02dde0 con 0x7fb6340c7ac0 2024-02-21T23:53:38.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.898+0000 7fb631ffb640 1 -- 172.21.15.133:0/3893211408 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c035660 con 0x7fb6340c7ac0 2024-02-21T23:53:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.898+0000 7fb63946d640 1 -- 172.21.15.133:0/3893211408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6340c7ac0 msgr2=0x7fb63406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.898+0000 7fb63946d640 1 --2- 172.21.15.133:0/3893211408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6340c7ac0 0x7fb63406e8b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb6340726c0 tx=0x7fb62c02d030 comp rx=0 tx=0).stop 2024-02-21T23:53:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.899+0000 7fb63946d640 1 -- 172.21.15.133:0/3893211408 shutdown_connections 2024-02-21T23:53:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.899+0000 7fb63946d640 1 --2- 172.21.15.133:0/3893211408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb6340c7ac0 0x7fb63406e8b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.899+0000 7fb63946d640 1 --2- 172.21.15.133:0/3893211408 >> v2:172.21.15.142:3300/0 conn(0x7fb634071f60 0x7fb6340c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.899+0000 7fb63946d640 1 -- 172.21.15.133:0/3893211408 >> 172.21.15.133:0/3893211408 conn(0x7fb63406aec0 msgr2=0x7fb63406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.899+0000 7fb63946d640 1 -- 172.21.15.133:0/3893211408 shutdown_connections 2024-02-21T23:53:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.899+0000 7fb63946d640 1 -- 172.21.15.133:0/3893211408 wait complete. 2024-02-21T23:53:38.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.900+0000 7fb63946d640 1 Processor -- start 2024-02-21T23:53:38.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.900+0000 7fb63946d640 1 -- start start 2024-02-21T23:53:38.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.901+0000 7fb63946d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb634071f60 0x7fb63414add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.901+0000 7fb63946d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6340c7ac0 0x7fb63414ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.901+0000 7fb63946d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb634150a30 0x7fb63414d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.901+0000 7fb63946d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb634072a70 con 0x7fb634071f60 2024-02-21T23:53:38.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.901+0000 7fb63946d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb634072790 con 0x7fb6340c7ac0 2024-02-21T23:53:38.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.901+0000 7fb63946d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb634150e10 con 0x7fb634150a30 2024-02-21T23:53:38.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.901+0000 7fb6327fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6340c7ac0 0x7fb63414ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.901+0000 7fb6327fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6340c7ac0 0x7fb63414ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:59010/0 (socket says 172.21.15.133:59010) 2024-02-21T23:53:38.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.901+0000 7fb6327fc640 1 -- 172.21.15.133:0/4254463888 learned_addr learned my addr 172.21.15.133:0/4254463888 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:38.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.902+0000 7fb632ffd640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb634071f60 0x7fb63414add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.903+0000 7fb6337fe640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb634150a30 0x7fb63414d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.905+0000 7fb6327fc640 1 -- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb634150a30 msgr2=0x7fb63414d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.905+0000 7fb6327fc640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb634150a30 0x7fb63414d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.905+0000 7fb6327fc640 1 -- 172.21.15.133:0/4254463888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb634071f60 msgr2=0x7fb63414add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.905+0000 7fb6327fc640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb634071f60 0x7fb63414add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.905+0000 7fb6327fc640 1 -- 172.21.15.133:0/4254463888 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63414c720 con 0x7fb6340c7ac0 2024-02-21T23:53:38.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.907+0000 7fb6327fc640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6340c7ac0 0x7fb63414ccb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb62800e9b0 tx=0x7fb62800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.910+0000 7fb61b7fe640 1 -- 172.21.15.133:0/4254463888 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb628017070 con 0x7fb6340c7ac0 2024-02-21T23:53:38.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.910+0000 7fb61b7fe640 1 -- 172.21.15.133:0/4254463888 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62800c9a0 con 0x7fb6340c7ac0 2024-02-21T23:53:38.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.910+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6340c70d0 con 0x7fb6340c7ac0 2024-02-21T23:53:38.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.910+0000 7fb61b7fe640 1 -- 172.21.15.133:0/4254463888 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62800cd90 con 0x7fb6340c7ac0 2024-02-21T23:53:38.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.910+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb634149fc0 con 0x7fb6340c7ac0 2024-02-21T23:53:38.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.911+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb63406a130 con 0x7fb6340c7ac0 2024-02-21T23:53:38.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.930+0000 7fb61b7fe640 1 -- 172.21.15.133:0/4254463888 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb62801a070 con 0x7fb6340c7ac0 2024-02-21T23:53:38.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.931+0000 7fb61b7fe640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb62006b520 0x7fb62006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.931+0000 7fb632ffd640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb62006b520 0x7fb62006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.932+0000 7fb632ffd640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb62006b520 0x7fb62006d9e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb624007fd0 tx=0x7fb624007e80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.936+0000 7fb61b7fe640 1 -- 172.21.15.133:0/4254463888 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fb628057200 con 0x7fb6340c7ac0 2024-02-21T23:53:38.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.936+0000 7fb61b7fe640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fb620075040 0x7fb6200774e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:38.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.936+0000 7fb61b7fe640 1 -- 172.21.15.133:0/4254463888 --> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb620077bb0 con 0x7fb620075040 2024-02-21T23:53:38.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.937+0000 7fb6337fe640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fb620075040 0x7fb6200774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:38.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.937+0000 7fb6337fe640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fb620075040 0x7fb6200774e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:38.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.938+0000 7fb61b7fe640 1 -- 172.21.15.133:0/4254463888 <== osd.6 v2:172.21.15.142:6802/4241695382 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb620077bb0 con 0x7fb620075040 2024-02-21T23:53:38.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.964+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 --> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb6340c78a0 con 0x7fb620075040 2024-02-21T23:53:38.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.964+0000 7fb61b7fe640 1 -- 172.21.15.133:0/4254463888 <== osd.6 v2:172.21.15.142:6802/4241695382 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb6340c78a0 con 0x7fb620075040 2024-02-21T23:53:38.960 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:53:38.960 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-21T23:53:38.960 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:53:38.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fb620075040 msgr2=0x7fb6200774e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6802/4241695382,v1:172.21.15.142:6803/4241695382] conn(0x7fb620075040 0x7fb6200774e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:38.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb62006b520 msgr2=0x7fb62006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fb62006b520 0x7fb62006d9e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb624007fd0 tx=0x7fb624007e80 comp rx=0 tx=0).stop 2024-02-21T23:53:38.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6340c7ac0 msgr2=0x7fb63414ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:38.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 --2- 172.21.15.133:0/4254463888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6340c7ac0 0x7fb63414ccb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb62800e9b0 tx=0x7fb62800ee70 comp rx=0 tx=0).stop 2024-02-21T23:53:38.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb6337fe640 1 -- 172.21.15.133:0/4254463888 reap_dead start 2024-02-21T23:53:38.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 shutdown_connections 2024-02-21T23:53:38.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 >> 172.21.15.133:0/4254463888 conn(0x7fb63406aec0 msgr2=0x7fb63406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:38.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 shutdown_connections 2024-02-21T23:53:38.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:38.965+0000 7fb63946d640 1 -- 172.21.15.133:0/4254463888 wait complete. 2024-02-21T23:53:38.971 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-02-21T23:53:39.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.078+0000 7f05f4f08640 1 Processor -- start 2024-02-21T23:53:39.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.078+0000 7f05f4f08640 1 -- start start 2024-02-21T23:53:39.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.079+0000 7f05f4f08640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f00c53a0 0x7f05f0071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.079+0000 7f05f4f08640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f0158230 con 0x7f05f00c4fc0 2024-02-21T23:53:39.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.079+0000 7f05f4f08640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f0158510 con 0x7f05f015b7b0 2024-02-21T23:53:39.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.079+0000 7f05f4f08640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f0158eb0 con 0x7f05f00c53a0 2024-02-21T23:53:39.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.079+0000 7f05ee59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f05f00c4fc0 0x7f05f006f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54068/0 (socket says 172.21.15.133:54068) 2024-02-21T23:53:39.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.079+0000 7f05ee59c640 1 -- 172.21.15.133:0/1157762867 learned_addr learned my addr 172.21.15.133:0/1157762867 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:39.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.079+0000 7f05eed9d640 1 --1- 172.21.15.133:0/1157762867 >> v1:172.21.15.142:6789/0 conn(0x7f05f015b7b0 0x7f05f015bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57560/0 (socket says 172.21.15.133:57560) 2024-02-21T23:53:39.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.080+0000 7f05e7fff640 1 --2- 172.21.15.133:0/1157762867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f00c53a0 0x7f05f0071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.081+0000 7f05edd9b640 1 -- 172.21.15.133:0/1157762867 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959680732 0 0) 0x7f05f0158510 con 0x7f05f015b7b0 2024-02-21T23:53:39.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.081+0000 7f05edd9b640 1 -- 172.21.15.133:0/1157762867 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05dc003520 con 0x7f05f015b7b0 2024-02-21T23:53:39.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.082+0000 7f05edd9b640 1 -- 172.21.15.133:0/1157762867 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772733328 0 0) 0x7f05f0158230 con 0x7f05f00c4fc0 2024-02-21T23:53:39.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.082+0000 7f05edd9b640 1 -- 172.21.15.133:0/1157762867 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f0158510 con 0x7f05f00c4fc0 2024-02-21T23:53:39.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.082+0000 7f05e7fff640 1 -- 172.21.15.133:0/1157762867 >> v1:172.21.15.142:6789/0 conn(0x7f05f015b7b0 legacy=0x7f05f015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.082+0000 7f05e7fff640 1 -- 172.21.15.133:0/1157762867 >> v1:172.21.15.133:6789/0 conn(0x7f05f00c4fc0 legacy=0x7f05f006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.082+0000 7f05e7fff640 1 -- 172.21.15.133:0/1157762867 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f01587f0 con 0x7f05f00c53a0 2024-02-21T23:53:39.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.085+0000 7f05e7fff640 1 --2- 172.21.15.133:0/1157762867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f00c53a0 0x7f05f0071b60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05d80097d0 tx=0x7f05d802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a96cf317b9e60645 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:39.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.088+0000 7f05edd9b640 1 -- 172.21.15.133:0/1157762867 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8034070 con 0x7f05f00c53a0 2024-02-21T23:53:39.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.088+0000 7f05edd9b640 1 -- 172.21.15.133:0/1157762867 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d8030880 con 0x7f05f00c53a0 2024-02-21T23:53:39.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.088+0000 7f05edd9b640 1 -- 172.21.15.133:0/1157762867 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8030c50 con 0x7f05f00c53a0 2024-02-21T23:53:39.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.089+0000 7f05f4f08640 1 -- 172.21.15.133:0/1157762867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f00c53a0 msgr2=0x7f05f0071b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.089+0000 7f05f4f08640 1 --2- 172.21.15.133:0/1157762867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f00c53a0 0x7f05f0071b60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05d80097d0 tx=0x7f05d802b9c0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.089+0000 7f05f4f08640 1 -- 172.21.15.133:0/1157762867 shutdown_connections 2024-02-21T23:53:39.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.089+0000 7f05f4f08640 1 --2- 172.21.15.133:0/1157762867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f00c53a0 0x7f05f0071b60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.089+0000 7f05f4f08640 1 -- 172.21.15.133:0/1157762867 >> 172.21.15.133:0/1157762867 conn(0x7f05f006aec0 msgr2=0x7f05f006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:39.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.089+0000 7f05f4f08640 1 -- 172.21.15.133:0/1157762867 shutdown_connections 2024-02-21T23:53:39.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.089+0000 7f05f4f08640 1 -- 172.21.15.133:0/1157762867 wait complete. 2024-02-21T23:53:39.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.090+0000 7f05f4f08640 1 Processor -- start 2024-02-21T23:53:39.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.091+0000 7f05f4f08640 1 -- start start 2024-02-21T23:53:39.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.091+0000 7f05f4f08640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05f00c4fc0 0x7f05f0149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.091+0000 7f05f4f08640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f00c53a0 0x7f05f01edef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.091+0000 7f05f4f08640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f015b7b0 0x7f05f01ee430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.091+0000 7f05f4f08640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f006f8d0 con 0x7f05f00c53a0 2024-02-21T23:53:39.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.091+0000 7f05f4f08640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f0157f20 con 0x7f05f00c4fc0 2024-02-21T23:53:39.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.091+0000 7f05f4f08640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f006fa50 con 0x7f05f015b7b0 2024-02-21T23:53:39.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.092+0000 7f05e7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f00c53a0 0x7f05f01edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.092+0000 7f05eed9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f015b7b0 0x7f05f01ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.092+0000 7f05ee59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05f00c4fc0 0x7f05f0149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.092+0000 7f05e7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f00c53a0 0x7f05f01edef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36380/0 (socket says 172.21.15.133:36380) 2024-02-21T23:53:39.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.092+0000 7f05e7fff640 1 -- 172.21.15.133:0/1398675248 learned_addr learned my addr 172.21.15.133:0/1398675248 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:39.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.095+0000 7f05eed9d640 1 -- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05f00c4fc0 msgr2=0x7f05f0149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.095+0000 7f05eed9d640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05f00c4fc0 0x7f05f0149ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.095+0000 7f05eed9d640 1 -- 172.21.15.133:0/1398675248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f00c53a0 msgr2=0x7f05f01edef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.095+0000 7f05eed9d640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f00c53a0 0x7f05f01edef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.095+0000 7f05eed9d640 1 -- 172.21.15.133:0/1398675248 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f00cb9e0 con 0x7f05f015b7b0 2024-02-21T23:53:39.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.098+0000 7f05eed9d640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f015b7b0 0x7f05f01ee430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f05e8001560 tx=0x7f05e800cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:39.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.101+0000 7f05e6ffd640 1 -- 172.21.15.133:0/1398675248 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8018070 con 0x7f05f015b7b0 2024-02-21T23:53:39.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.102+0000 7f05e6ffd640 1 -- 172.21.15.133:0/1398675248 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8004d80 con 0x7f05f015b7b0 2024-02-21T23:53:39.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.102+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f014c230 con 0x7f05f015b7b0 2024-02-21T23:53:39.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.102+0000 7f05e6ffd640 1 -- 172.21.15.133:0/1398675248 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e800a880 con 0x7f05f015b7b0 2024-02-21T23:53:39.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.102+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f00cbc50 con 0x7f05f015b7b0 2024-02-21T23:53:39.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.103+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f015c350 con 0x7f05f015b7b0 2024-02-21T23:53:39.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.121+0000 7f05e6ffd640 1 -- 172.21.15.133:0/1398675248 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05e801b070 con 0x7f05f015b7b0 2024-02-21T23:53:39.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.121+0000 7f05e6ffd640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05d006b520 0x7f05d006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.122+0000 7f05ee59c640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05d006b520 0x7f05d006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.122+0000 7f05ee59c640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05d006b520 0x7f05d006d9e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005880 tx=0x7f05e0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:39.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.125+0000 7f05e6ffd640 1 -- 172.21.15.133:0/1398675248 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f05e8056410 con 0x7f05f015b7b0 2024-02-21T23:53:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.372+0000 7f05e6ffd640 1 -- 172.21.15.133:0/1398675248 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f05f015c350 con 0x7f05f015b7b0 2024-02-21T23:53:39.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.553+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f05f006a130 con 0x7f05d006b520 2024-02-21T23:53:39.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.554+0000 7f05e6ffd640 1 -- 172.21.15.133:0/1398675248 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f05f006a130 con 0x7f05d006b520 2024-02-21T23:53:39.549 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 6 to deep-scrub 2024-02-21T23:53:39.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.555+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05d006b520 msgr2=0x7f05d006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.555+0000 7f05f4f08640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05d006b520 0x7f05d006d9e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005880 tx=0x7f05e0009040 comp rx=0 tx=0).stop 2024-02-21T23:53:39.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.555+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f015b7b0 msgr2=0x7f05f01ee430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.555+0000 7f05f4f08640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f015b7b0 0x7f05f01ee430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f05e8001560 tx=0x7f05e800cb90 comp rx=0 tx=0).stop 2024-02-21T23:53:39.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.555+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 shutdown_connections 2024-02-21T23:53:39.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.555+0000 7f05f4f08640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f05d006b520 0x7f05d006d9e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.555+0000 7f05f4f08640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05f015b7b0 0x7f05f01ee430 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.555+0000 7f05f4f08640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f00c53a0 0x7f05f01edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.555+0000 7f05f4f08640 1 --2- 172.21.15.133:0/1398675248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05f00c4fc0 0x7f05f0149ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.556+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 >> 172.21.15.133:0/1398675248 conn(0x7f05f006aec0 msgr2=0x7f05f01572c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.556+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 shutdown_connections 2024-02-21T23:53:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.556+0000 7f05f4f08640 1 -- 172.21.15.133:0/1398675248 wait complete. 2024-02-21T23:53:39.562 INFO:tasks.ceph:Scrubbing osd.7 2024-02-21T23:53:39.562 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-02-21T23:53:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.672+0000 7fa174b9c640 1 Processor -- start 2024-02-21T23:53:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.672+0000 7fa174b9c640 1 -- start start 2024-02-21T23:53:39.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.673+0000 7fa174b9c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa1700c73d0 0x7fa17006aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.673+0000 7fa174b9c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa17006d0a0 0x7fa17015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.673+0000 7fa174b9c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17006e660 con 0x7fa170152270 2024-02-21T23:53:39.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.673+0000 7fa174b9c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17015b470 con 0x7fa1700c73d0 2024-02-21T23:53:39.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.673+0000 7fa174b9c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa170152650 con 0x7fa17006d0a0 2024-02-21T23:53:39.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.673+0000 7fa16dd9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa1700c73d0 0x7fa17006aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.674+0000 7fa16ed9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa17006d0a0 0x7fa17015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.674+0000 7fa16e59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa170152270 0x7fa170072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54082/0 (socket says 172.21.15.133:54082) 2024-02-21T23:53:39.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.674+0000 7fa16e59c640 1 -- 172.21.15.133:0/3592134254 learned_addr learned my addr 172.21.15.133:0/3592134254 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:39.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.677+0000 7fa16dd9b640 1 -- 172.21.15.133:0/3592134254 >> v2:172.21.15.142:3300/0 conn(0x7fa17006d0a0 msgr2=0x7fa17015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:53:39.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.677+0000 7fa16dd9b640 1 --2- 172.21.15.133:0/3592134254 >> v2:172.21.15.142:3300/0 conn(0x7fa17006d0a0 0x7fa17015bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.677+0000 7fa16dd9b640 1 -- 172.21.15.133:0/3592134254 >> v1:172.21.15.133:6789/0 conn(0x7fa170152270 legacy=0x7fa170072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.677+0000 7fa16dd9b640 1 -- 172.21.15.133:0/3592134254 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17006e940 con 0x7fa1700c73d0 2024-02-21T23:53:39.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.681+0000 7fa16dd9b640 1 --2- 172.21.15.133:0/3592134254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1700c73d0 0x7fa17006aec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa1700719c0 tx=0x7fa16402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7596229ed11f1fd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:39.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.685+0000 7fa16d59a640 1 -- 172.21.15.133:0/3592134254 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164038070 con 0x7fa1700c73d0 2024-02-21T23:53:39.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.685+0000 7fa16d59a640 1 -- 172.21.15.133:0/3592134254 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164031710 con 0x7fa1700c73d0 2024-02-21T23:53:39.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.685+0000 7fa16d59a640 1 -- 172.21.15.133:0/3592134254 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164031ae0 con 0x7fa1700c73d0 2024-02-21T23:53:39.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.686+0000 7fa174b9c640 1 -- 172.21.15.133:0/3592134254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1700c73d0 msgr2=0x7fa17006aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.686+0000 7fa174b9c640 1 --2- 172.21.15.133:0/3592134254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1700c73d0 0x7fa17006aec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa1700719c0 tx=0x7fa16402b7b0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.686+0000 7fa174b9c640 1 -- 172.21.15.133:0/3592134254 shutdown_connections 2024-02-21T23:53:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.686+0000 7fa174b9c640 1 --2- 172.21.15.133:0/3592134254 >> v2:172.21.15.142:3300/0 conn(0x7fa17006d0a0 0x7fa17015bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.686+0000 7fa174b9c640 1 --2- 172.21.15.133:0/3592134254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1700c73d0 0x7fa17006aec0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.686+0000 7fa174b9c640 1 -- 172.21.15.133:0/3592134254 >> 172.21.15.133:0/3592134254 conn(0x7fa1700cc620 msgr2=0x7fa1700c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.686+0000 7fa174b9c640 1 -- 172.21.15.133:0/3592134254 shutdown_connections 2024-02-21T23:53:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.686+0000 7fa174b9c640 1 -- 172.21.15.133:0/3592134254 wait complete. 2024-02-21T23:53:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.688+0000 7fa174b9c640 1 Processor -- start 2024-02-21T23:53:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.688+0000 7fa174b9c640 1 -- start start 2024-02-21T23:53:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.688+0000 7fa174b9c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa17006d0a0 0x7fa1701581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.688+0000 7fa174b9c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1700c73d0 0x7fa170154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.688+0000 7fa174b9c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa170152270 0x7fa170155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.688+0000 7fa174b9c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17015b9a0 con 0x7fa17006d0a0 2024-02-21T23:53:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.688+0000 7fa174b9c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17015b6c0 con 0x7fa170152270 2024-02-21T23:53:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.688+0000 7fa174b9c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17006c410 con 0x7fa1700c73d0 2024-02-21T23:53:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.689+0000 7fa16e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa17006d0a0 0x7fa1701581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.689+0000 7fa16ed9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa170152270 0x7fa170155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.689+0000 7fa16dd9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1700c73d0 0x7fa170154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.689+0000 7fa16e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa17006d0a0 0x7fa1701581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36396/0 (socket says 172.21.15.133:36396) 2024-02-21T23:53:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.689+0000 7fa16e59c640 1 -- 172.21.15.133:0/3925285281 learned_addr learned my addr 172.21.15.133:0/3925285281 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:39.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.692+0000 7fa16e59c640 1 -- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1700c73d0 msgr2=0x7fa170154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.692+0000 7fa16e59c640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1700c73d0 0x7fa170154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.692+0000 7fa16e59c640 1 -- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa170152270 msgr2=0x7fa170155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.692+0000 7fa16e59c640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa170152270 0x7fa170155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.692+0000 7fa16e59c640 1 -- 172.21.15.133:0/3925285281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1701543e0 con 0x7fa17006d0a0 2024-02-21T23:53:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.695+0000 7fa16e59c640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa17006d0a0 0x7fa1701581d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa16000c400 tx=0x7fa16000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:39.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.698+0000 7fa15effd640 1 -- 172.21.15.133:0/3925285281 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160017070 con 0x7fa17006d0a0 2024-02-21T23:53:39.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.698+0000 7fa15effd640 1 -- 172.21.15.133:0/3925285281 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160005ce0 con 0x7fa17006d0a0 2024-02-21T23:53:39.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.698+0000 7fa15effd640 1 -- 172.21.15.133:0/3925285281 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600072f0 con 0x7fa17006d0a0 2024-02-21T23:53:39.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.698+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa170154c40 con 0x7fa17006d0a0 2024-02-21T23:53:39.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.698+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa170154960 con 0x7fa17006d0a0 2024-02-21T23:53:39.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.699+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1700cbb00 con 0x7fa17006d0a0 2024-02-21T23:53:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.718+0000 7fa15effd640 1 -- 172.21.15.133:0/3925285281 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa16001a070 con 0x7fa17006d0a0 2024-02-21T23:53:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.719+0000 7fa15effd640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa15006b520 0x7fa15006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.719+0000 7fa16dd9b640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa15006b520 0x7fa15006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.720+0000 7fa16dd9b640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa15006b520 0x7fa15006d9e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa164035000 tx=0x7fa164008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:39.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.722+0000 7fa15effd640 1 -- 172.21.15.133:0/3925285281 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fa160056550 con 0x7fa17006d0a0 2024-02-21T23:53:39.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.722+0000 7fa15effd640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7fa1500750c0 0x7fa150077560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.722+0000 7fa15effd640 1 -- 172.21.15.133:0/3925285281 --> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa150077c30 con 0x7fa1500750c0 2024-02-21T23:53:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.722+0000 7fa15effd640 1 -- 172.21.15.133:0/3925285281 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa160054750 con 0x7fa17006d0a0 2024-02-21T23:53:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.722+0000 7fa16ed9d640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7fa1500750c0 0x7fa150077560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.722+0000 7fa16ed9d640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7fa1500750c0 0x7fa150077560 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.723+0000 7fa15effd640 1 -- 172.21.15.133:0/3925285281 <== osd.7 v2:172.21.15.142:6810/3083092904 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa150077c30 con 0x7fa1500750c0 2024-02-21T23:53:39.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 --> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa17006a130 con 0x7fa1500750c0 2024-02-21T23:53:39.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa15effd640 1 -- 172.21.15.133:0/3925285281 <== osd.7 v2:172.21.15.142:6810/3083092904 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa17006a130 con 0x7fa1500750c0 2024-02-21T23:53:39.731 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-02-21T23:53:39.731 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-21T23:53:39.731 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-02-21T23:53:39.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7fa1500750c0 msgr2=0x7fa150077560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6810/3083092904,v1:172.21.15.142:6811/3083092904] conn(0x7fa1500750c0 0x7fa150077560 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa15006b520 msgr2=0x7fa15006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa15006b520 0x7fa15006d9e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa164035000 tx=0x7fa164008030 comp rx=0 tx=0).stop 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa17006d0a0 msgr2=0x7fa1701581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 --2- 172.21.15.133:0/3925285281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa17006d0a0 0x7fa1701581d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa16000c400 tx=0x7fa16000c8c0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa16ed9d640 1 -- 172.21.15.133:0/3925285281 reap_dead start 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 shutdown_connections 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 >> 172.21.15.133:0/3925285281 conn(0x7fa1700cc620 msgr2=0x7fa1700c9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 shutdown_connections 2024-02-21T23:53:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.736+0000 7fa174b9c640 1 -- 172.21.15.133:0/3925285281 wait complete. 2024-02-21T23:53:39.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-02-21T23:53:39.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.849+0000 7fa72a571640 1 Processor -- start 2024-02-21T23:53:39.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.849+0000 7fa72a571640 1 -- start start 2024-02-21T23:53:39.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.849+0000 7fa72a571640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa7240c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.849+0000 7fa72a571640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72415a720 con 0x7fa72415a290 2024-02-21T23:53:39.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.850+0000 7fa72a571640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72415a9e0 con 0x7fa724154450 2024-02-21T23:53:39.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.850+0000 7fa72a571640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72406c2f0 con 0x7fa7240c3640 2024-02-21T23:53:39.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.850+0000 7fa7237fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa72415a290 0x7fa7240c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54094/0 (socket says 172.21.15.133:54094) 2024-02-21T23:53:39.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.850+0000 7fa7237fe640 1 -- 172.21.15.133:0/592427 learned_addr learned my addr 172.21.15.133:0/592427 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:39.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.850+0000 7fa728b0e640 1 --2- 172.21.15.133:0/592427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa7240c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.853+0000 7fa722ffd640 1 -- 172.21.15.133:0/592427 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869168829 0 0) 0x7fa72415a720 con 0x7fa72415a290 2024-02-21T23:53:39.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.853+0000 7fa722ffd640 1 -- 172.21.15.133:0/592427 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa70c003520 con 0x7fa72415a290 2024-02-21T23:53:39.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.853+0000 7fa722ffd640 1 -- 172.21.15.133:0/592427 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567156836 0 0) 0x7fa72415a9e0 con 0x7fa724154450 2024-02-21T23:53:39.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.853+0000 7fa722ffd640 1 -- 172.21.15.133:0/592427 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa72415a720 con 0x7fa724154450 2024-02-21T23:53:39.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.853+0000 7fa728b0e640 1 -- 172.21.15.133:0/592427 >> v1:172.21.15.142:6789/0 conn(0x7fa724154450 legacy=0x7fa72406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.853+0000 7fa728b0e640 1 -- 172.21.15.133:0/592427 >> v1:172.21.15.133:6789/0 conn(0x7fa72415a290 legacy=0x7fa7240c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.853+0000 7fa728b0e640 1 -- 172.21.15.133:0/592427 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72415acc0 con 0x7fa7240c3640 2024-02-21T23:53:39.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.856+0000 7fa728b0e640 1 --2- 172.21.15.133:0/592427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa7240c3ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009dc0 tx=0x7fa71c02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=447c275d540944ba server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:39.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa722ffd640 1 -- 172.21.15.133:0/592427 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c039070 con 0x7fa7240c3640 2024-02-21T23:53:39.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa722ffd640 1 -- 172.21.15.133:0/592427 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c02cd50 con 0x7fa7240c3640 2024-02-21T23:53:39.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa722ffd640 1 -- 172.21.15.133:0/592427 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c034660 con 0x7fa7240c3640 2024-02-21T23:53:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa72a571640 1 -- 172.21.15.133:0/592427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 msgr2=0x7fa7240c3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa72a571640 1 --2- 172.21.15.133:0/592427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa7240c3ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009dc0 tx=0x7fa71c02bfa0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa72a571640 1 -- 172.21.15.133:0/592427 shutdown_connections 2024-02-21T23:53:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa72a571640 1 --2- 172.21.15.133:0/592427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa7240c3ab0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa72a571640 1 -- 172.21.15.133:0/592427 >> 172.21.15.133:0/592427 conn(0x7fa724070ec0 msgr2=0x7fa7240712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa72a571640 1 -- 172.21.15.133:0/592427 shutdown_connections 2024-02-21T23:53:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.860+0000 7fa72a571640 1 -- 172.21.15.133:0/592427 wait complete. 2024-02-21T23:53:39.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.861+0000 7fa72a571640 1 Processor -- start 2024-02-21T23:53:39.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.862+0000 7fa72a571640 1 -- start start 2024-02-21T23:53:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.862+0000 7fa72a571640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa72414ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.862+0000 7fa72a571640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa724154450 0x7fa724157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.862+0000 7fa72a571640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7241f3430 0x7fa724154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.862+0000 7fa72a571640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240700d0 con 0x7fa724154450 2024-02-21T23:53:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.862+0000 7fa72a571640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72406f4a0 con 0x7fa7241f3430 2024-02-21T23:53:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.862+0000 7fa72a571640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa724070250 con 0x7fa7240c3640 2024-02-21T23:53:39.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.863+0000 7fa7237fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa724154450 0x7fa724157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.863+0000 7fa723fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa72414ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.863+0000 7fa728b0e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7241f3430 0x7fa724154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:39.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.863+0000 7fa723fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa72414ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:48090/0 (socket says 172.21.15.133:48090) 2024-02-21T23:53:39.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.863+0000 7fa723fff640 1 -- 172.21.15.133:0/3136571892 learned_addr learned my addr 172.21.15.133:0/3136571892 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.866+0000 7fa7237fe640 1 -- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 msgr2=0x7fa72414ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.866+0000 7fa7237fe640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa72414ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.866+0000 7fa7237fe640 1 -- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7241f3430 msgr2=0x7fa724154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.866+0000 7fa7237fe640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7241f3430 0x7fa724154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.867+0000 7fa7237fe640 1 -- 172.21.15.133:0/3136571892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724153520 con 0x7fa724154450 2024-02-21T23:53:39.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.869+0000 7fa7237fe640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa724154450 0x7fa724157880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa718001200 tx=0x7fa71800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:39.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.872+0000 7fa720ff9640 1 -- 172.21.15.133:0/3136571892 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718017070 con 0x7fa724154450 2024-02-21T23:53:39.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.872+0000 7fa720ff9640 1 -- 172.21.15.133:0/3136571892 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa718005ce0 con 0x7fa724154450 2024-02-21T23:53:39.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.872+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7241f4060 con 0x7fa724154450 2024-02-21T23:53:39.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.872+0000 7fa720ff9640 1 -- 172.21.15.133:0/3136571892 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180072d0 con 0x7fa724154450 2024-02-21T23:53:39.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.872+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7241f3d80 con 0x7fa724154450 2024-02-21T23:53:39.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:39.873+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72406a130 con 0x7fa724154450 2024-02-21T23:53:40.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.119+0000 7fa720ff9640 1 -- 172.21.15.133:0/3136571892 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa71801a070 con 0x7fa724154450 2024-02-21T23:53:40.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.120+0000 7fa720ff9640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa71006b550 0x7fa71006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:40.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.120+0000 7fa723fff640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa71006b550 0x7fa71006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:40.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.120+0000 7fa723fff640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa71006b550 0x7fa71006da10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa714007fd0 tx=0x7fa714009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:40.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.134+0000 7fa720ff9640 1 -- 172.21.15.133:0/3136571892 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fa718056170 con 0x7fa724154450 2024-02-21T23:53:40.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.134+0000 7fa720ff9640 1 -- 172.21.15.133:0/3136571892 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fa7180580d0 con 0x7fa724154450 2024-02-21T23:53:40.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.316+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7241537e0 con 0x7fa71006b550 2024-02-21T23:53:40.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.317+0000 7fa720ff9640 1 -- 172.21.15.133:0/3136571892 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa7241537e0 con 0x7fa71006b550 2024-02-21T23:53:40.312 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 7 to deep-scrub 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa71006b550 msgr2=0x7fa71006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa71006b550 0x7fa71006da10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa714007fd0 tx=0x7fa714009040 comp rx=0 tx=0).stop 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa724154450 msgr2=0x7fa724157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa724154450 0x7fa724157880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa718001200 tx=0x7fa71800c890 comp rx=0 tx=0).stop 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 shutdown_connections 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fa71006b550 0x7fa71006da10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7241f3430 0x7fa724154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa724154450 0x7fa724157880 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 --2- 172.21.15.133:0/3136571892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7240c3640 0x7fa72414ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 >> 172.21.15.133:0/3136571892 conn(0x7fa724070ec0 msgr2=0x7fa7240ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 shutdown_connections 2024-02-21T23:53:40.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.319+0000 7fa72a571640 1 -- 172.21.15.133:0/3136571892 wait complete. 2024-02-21T23:53:40.326 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-21T23:53:40.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.432+0000 7fe30c711640 1 Processor -- start 2024-02-21T23:53:40.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.432+0000 7fe30c711640 1 -- start start 2024-02-21T23:53:40.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.433+0000 7fe30c711640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c41a0 0x7fe3040cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:40.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.433+0000 7fe30c711640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe304156e90 con 0x7fe304150d30 2024-02-21T23:53:40.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.433+0000 7fe30c711640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe304157150 con 0x7fe3040c3dc0 2024-02-21T23:53:40.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.433+0000 7fe30c711640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe304157430 con 0x7fe3040c41a0 2024-02-21T23:53:40.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.434+0000 7fe30a4ad640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fe3040c3dc0 0x7fe304070410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:57576/0 (socket says 172.21.15.133:57576) 2024-02-21T23:53:40.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.434+0000 7fe30a4ad640 1 -- 172.21.15.133:0/94527453 learned_addr learned my addr 172.21.15.133:0/94527453 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:53:40.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.434+0000 7fe309cac640 1 --2- 172.21.15.133:0/94527453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c41a0 0x7fe3040cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:40.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.436+0000 7fe3094ab640 1 -- 172.21.15.133:0/94527453 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62340087 0 0) 0x7fe304156e90 con 0x7fe304150d30 2024-02-21T23:53:40.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.436+0000 7fe3094ab640 1 -- 172.21.15.133:0/94527453 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2f0003520 con 0x7fe304150d30 2024-02-21T23:53:40.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.436+0000 7fe3094ab640 1 -- 172.21.15.133:0/94527453 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265380788 0 0) 0x7fe304157150 con 0x7fe3040c3dc0 2024-02-21T23:53:40.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.436+0000 7fe3094ab640 1 -- 172.21.15.133:0/94527453 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe304156e90 con 0x7fe3040c3dc0 2024-02-21T23:53:40.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.436+0000 7fe309cac640 1 -- 172.21.15.133:0/94527453 >> v1:172.21.15.142:6789/0 conn(0x7fe3040c3dc0 legacy=0x7fe304070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:40.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.436+0000 7fe309cac640 1 -- 172.21.15.133:0/94527453 >> v1:172.21.15.133:6789/0 conn(0x7fe304150d30 legacy=0x7fe30415a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:40.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.436+0000 7fe309cac640 1 -- 172.21.15.133:0/94527453 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe304157710 con 0x7fe3040c41a0 2024-02-21T23:53:40.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.438+0000 7fe309cac640 1 --2- 172.21.15.133:0/94527453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c41a0 0x7fe3040cd010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40097d0 tx=0x7fe2f402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473d675c03a54444 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.443+0000 7fe3094ab640 1 -- 172.21.15.133:0/94527453 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4035020 con 0x7fe3040c41a0 2024-02-21T23:53:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.443+0000 7fe3094ab640 1 -- 172.21.15.133:0/94527453 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f4031890 con 0x7fe3040c41a0 2024-02-21T23:53:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.443+0000 7fe3094ab640 1 -- 172.21.15.133:0/94527453 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4031c60 con 0x7fe3040c41a0 2024-02-21T23:53:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.443+0000 7fe30c711640 1 -- 172.21.15.133:0/94527453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c41a0 msgr2=0x7fe3040cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.443+0000 7fe30c711640 1 --2- 172.21.15.133:0/94527453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c41a0 0x7fe3040cd010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40097d0 tx=0x7fe2f402b930 comp rx=0 tx=0).stop 2024-02-21T23:53:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.444+0000 7fe30c711640 1 -- 172.21.15.133:0/94527453 shutdown_connections 2024-02-21T23:53:40.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.444+0000 7fe30c711640 1 --2- 172.21.15.133:0/94527453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c41a0 0x7fe3040cd010 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.444+0000 7fe30c711640 1 -- 172.21.15.133:0/94527453 >> 172.21.15.133:0/94527453 conn(0x7fe304061b50 msgr2=0x7fe304072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:40.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.444+0000 7fe30c711640 1 -- 172.21.15.133:0/94527453 shutdown_connections 2024-02-21T23:53:40.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.444+0000 7fe30c711640 1 -- 172.21.15.133:0/94527453 wait complete. 2024-02-21T23:53:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.445+0000 7fe30c711640 1 Processor -- start 2024-02-21T23:53:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.445+0000 7fe30c711640 1 -- start start 2024-02-21T23:53:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.445+0000 7fe30c711640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c3dc0 0x7fe3040cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.445+0000 7fe30c711640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3040c41a0 0x7fe3040c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.445+0000 7fe30c711640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe304150d30 0x7fe304151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.445+0000 7fe30c711640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304070190 con 0x7fe304150d30 2024-02-21T23:53:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.445+0000 7fe30c711640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304157cd0 con 0x7fe3040c41a0 2024-02-21T23:53:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.445+0000 7fe30c711640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30415ad10 con 0x7fe3040c3dc0 2024-02-21T23:53:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.446+0000 7fe30acae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe304150d30 0x7fe304151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.446+0000 7fe30a4ad640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c3dc0 0x7fe3040cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.446+0000 7fe30acae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe304150d30 0x7fe304151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36410/0 (socket says 172.21.15.133:36410) 2024-02-21T23:53:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.446+0000 7fe30a4ad640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c3dc0 0x7fe3040cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:48102/0 (socket says 172.21.15.133:48102) 2024-02-21T23:53:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.446+0000 7fe30acae640 1 -- 172.21.15.133:0/2295939666 learned_addr learned my addr 172.21.15.133:0/2295939666 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:53:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.447+0000 7fe309cac640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3040c41a0 0x7fe3040c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:40.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.449+0000 7fe30acae640 1 -- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c3dc0 msgr2=0x7fe3040cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:40.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.449+0000 7fe30acae640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c3dc0 0x7fe3040cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.449+0000 7fe30acae640 1 -- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3040c41a0 msgr2=0x7fe3040c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:40.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.449+0000 7fe30acae640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3040c41a0 0x7fe3040c2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.449+0000 7fe30acae640 1 -- 172.21.15.133:0/2295939666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3040cc150 con 0x7fe304150d30 2024-02-21T23:53:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.452+0000 7fe30acae640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe304150d30 0x7fe304151690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c3c0 tx=0x7fe2fc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:40.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.462+0000 7fe2faffd640 1 -- 172.21.15.133:0/2295939666 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc017070 con 0x7fe304150d30 2024-02-21T23:53:40.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.462+0000 7fe2faffd640 1 -- 172.21.15.133:0/2295939666 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc0052d0 con 0x7fe304150d30 2024-02-21T23:53:40.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.462+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3040cb770 con 0x7fe304150d30 2024-02-21T23:53:40.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.462+0000 7fe2faffd640 1 -- 172.21.15.133:0/2295939666 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0037f0 con 0x7fe304150d30 2024-02-21T23:53:40.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.462+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3040cd0c0 con 0x7fe304150d30 2024-02-21T23:53:40.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.463+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30406a130 con 0x7fe304150d30 2024-02-21T23:53:40.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.482+0000 7fe2faffd640 1 -- 172.21.15.133:0/2295939666 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2fc01a070 con 0x7fe304150d30 2024-02-21T23:53:40.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.483+0000 7fe2faffd640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2e806b520 0x7fe2e806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:53:40.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.483+0000 7fe30a4ad640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2e806b520 0x7fe2e806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:53:40.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.483+0000 7fe30a4ad640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2e806b520 0x7fe2e806d9e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe300001560 tx=0x7fe300001a90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:53:40.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.485+0000 7fe2faffd640 1 -- 172.21.15.133:0/2295939666 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fe2fc0561c0 con 0x7fe304150d30 2024-02-21T23:53:40.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.727+0000 7fe2faffd640 1 -- 172.21.15.133:0/2295939666 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fe3040cc150 con 0x7fe304150d30 2024-02-21T23:53:40.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.899+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3040c8720 con 0x7fe2e806b520 2024-02-21T23:53:40.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.906+0000 7fe2faffd640 1 -- 172.21.15.133:0/2295939666 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+331395 (secure 0 0 0) 0x7fe3040c8720 con 0x7fe2e806b520 2024-02-21T23:53:40.905 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:53:40.907 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2e806b520 msgr2=0x7fe2e806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2e806b520 0x7fe2e806d9e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe300001560 tx=0x7fe300001a90 comp rx=0 tx=0).stop 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe304150d30 msgr2=0x7fe304151690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe304150d30 0x7fe304151690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c3c0 tx=0x7fe2fc001e70 comp rx=0 tx=0).stop 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 shutdown_connections 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fe2e806b520 0x7fe2e806d9e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe304150d30 0x7fe304151690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3040c41a0 0x7fe3040c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 --2- 172.21.15.133:0/2295939666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3040c3dc0 0x7fe3040cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 >> 172.21.15.133:0/2295939666 conn(0x7fe304061b50 msgr2=0x7fe3040c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:53:40.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 shutdown_connections 2024-02-21T23:53:40.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:53:40.910+0000 7fe30c711640 1 -- 172.21.15.133:0/2295939666 wait complete. 2024-02-21T23:53:40.917 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":877,"stamp":"2024-02-21T23:53:39.044295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":41122540,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":1110,"num_read_kb":1160,"num_write":15803,"num_write_kb":371997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16689,"ondisk_log_size":16689,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":426808,"kb_used_data":56632,"kb_used_omap":12,"kb_used_meta":370099,"kb_avail":754547912,"statfs":{"total":773094113280,"available":772657061888,"internally_reserved":0,"allocated":57991168,"data_stored":83162120,"data_compressed":7608710,"data_compressed_allocated":11812864,"data_compressed_original":38363136,"omap_allocated":12750,"internal_metadata":378981938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":949,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002552"},"pg_stats":[{"pgid":"2.1f","version":"26'607","reported_seq":655,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.605301+0000","last_change":"2024-02-21T23:53:38.605353+0000","last_active":"2024-02-21T23:53:38.605301+0000","last_peered":"2024-02-21T23:53:38.605301+0000","last_clean":"2024-02-21T23:53:38.605301+0000","last_became_active":"2024-02-21T23:25:16.134112+0000","last_became_peered":"2024-02-21T23:25:16.134112+0000","last_unstale":"2024-02-21T23:53:38.605301+0000","last_undegraded":"2024-02-21T23:53:38.605301+0000","last_fullsized":"2024-02-21T23:53:38.605301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'607","last_scrub_stamp":"2024-02-21T23:53:38.605182+0000","last_deep_scrub":"26'607","last_deep_scrub_stamp":"2024-02-21T23:53:38.605182+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.605182+0000","objects_scrubbed":3,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:43.294522+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'30","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471166+0000","last_change":"2024-02-21T23:25:17.135738+0000","last_active":"2024-02-21T23:53:24.471166+0000","last_peered":"2024-02-21T23:53:24.471166+0000","last_clean":"2024-02-21T23:53:24.471166+0000","last_became_active":"2024-02-21T23:25:17.135586+0000","last_became_peered":"2024-02-21T23:25:17.135586+0000","last_unstale":"2024-02-21T23:53:24.471166+0000","last_undegraded":"2024-02-21T23:53:24.471166+0000","last_fullsized":"2024-02-21T23:53:24.471166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'161","reported_seq":191,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.566668+0000","last_change":"2024-02-21T23:25:22.248962+0000","last_active":"2024-02-21T23:52:56.566668+0000","last_peered":"2024-02-21T23:52:56.566668+0000","last_clean":"2024-02-21T23:52:56.566668+0000","last_became_active":"2024-02-21T23:25:22.248610+0000","last_became_peered":"2024-02-21T23:25:22.248610+0000","last_unstale":"2024-02-21T23:52:56.566668+0000","last_undegraded":"2024-02-21T23:52:56.566668+0000","last_fullsized":"2024-02-21T23:52:56.566668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":10997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'750","reported_seq":791,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.577587+0000","last_change":"2024-02-21T23:25:16.123635+0000","last_active":"2024-02-21T23:53:18.577587+0000","last_peered":"2024-02-21T23:53:18.577587+0000","last_clean":"2024-02-21T23:53:18.577587+0000","last_became_active":"2024-02-21T23:25:16.123374+0000","last_became_peered":"2024-02-21T23:25:16.123374+0000","last_unstale":"2024-02-21T23:53:18.577587+0000","last_undegraded":"2024-02-21T23:53:18.577587+0000","last_fullsized":"2024-02-21T23:53:18.577587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":4534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'36","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604965+0000","last_change":"2024-02-21T23:25:17.192858+0000","last_active":"2024-02-21T23:53:24.604965+0000","last_peered":"2024-02-21T23:53:24.604965+0000","last_clean":"2024-02-21T23:53:24.604965+0000","last_became_active":"2024-02-21T23:25:17.192626+0000","last_became_peered":"2024-02-21T23:25:17.192626+0000","last_unstale":"2024-02-21T23:53:24.604965+0000","last_undegraded":"2024-02-21T23:53:24.604965+0000","last_fullsized":"2024-02-21T23:53:24.604965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'132","reported_seq":169,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.800608+0000","last_change":"2024-02-21T23:25:57.583837+0000","last_active":"2024-02-21T23:52:56.800608+0000","last_peered":"2024-02-21T23:52:56.800608+0000","last_clean":"2024-02-21T23:52:56.800608+0000","last_became_active":"2024-02-21T23:25:57.583416+0000","last_became_peered":"2024-02-21T23:25:57.583416+0000","last_unstale":"2024-02-21T23:52:56.800608+0000","last_undegraded":"2024-02-21T23:52:56.800608+0000","last_fullsized":"2024-02-21T23:52:56.800608+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":6267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"26'1343","reported_seq":1380,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.892783+0000","last_change":"2024-02-21T23:25:16.134805+0000","last_active":"2024-02-21T23:53:15.892783+0000","last_peered":"2024-02-21T23:53:15.892783+0000","last_clean":"2024-02-21T23:53:15.892783+0000","last_became_active":"2024-02-21T23:25:16.134533+0000","last_became_peered":"2024-02-21T23:25:16.134533+0000","last_unstale":"2024-02-21T23:53:15.892783+0000","last_undegraded":"2024-02-21T23:53:15.892783+0000","last_fullsized":"2024-02-21T23:53:15.892783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1334,"num_write_kb":8974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'32","reported_seq":67,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494748+0000","last_change":"2024-02-21T23:25:17.182131+0000","last_active":"2024-02-21T23:53:24.494748+0000","last_peered":"2024-02-21T23:53:24.494748+0000","last_clean":"2024-02-21T23:53:24.494748+0000","last_became_active":"2024-02-21T23:25:17.181886+0000","last_became_peered":"2024-02-21T23:25:17.181886+0000","last_unstale":"2024-02-21T23:53:24.494748+0000","last_undegraded":"2024-02-21T23:53:24.494748+0000","last_fullsized":"2024-02-21T23:53:24.494748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'141","reported_seq":171,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.694984+0000","last_change":"2024-02-21T23:25:21.683506+0000","last_active":"2024-02-21T23:52:56.694984+0000","last_peered":"2024-02-21T23:52:56.694984+0000","last_clean":"2024-02-21T23:52:56.694984+0000","last_became_active":"2024-02-21T23:25:21.683115+0000","last_became_peered":"2024-02-21T23:25:21.683115+0000","last_unstale":"2024-02-21T23:52:56.694984+0000","last_undegraded":"2024-02-21T23:52:56.694984+0000","last_fullsized":"2024-02-21T23:52:56.694984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":12261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"26'6","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:08.503461+0000","last_change":"2024-02-21T23:25:16.130851+0000","last_active":"2024-02-21T23:53:08.503461+0000","last_peered":"2024-02-21T23:53:08.503461+0000","last_clean":"2024-02-21T23:53:08.503461+0000","last_became_active":"2024-02-21T23:25:16.130604+0000","last_became_peered":"2024-02-21T23:25:16.130604+0000","last_unstale":"2024-02-21T23:53:08.503461+0000","last_undegraded":"2024-02-21T23:53:08.503461+0000","last_fullsized":"2024-02-21T23:53:08.503461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:39:32.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'47","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.495013+0000","last_change":"2024-02-21T23:25:17.192642+0000","last_active":"2024-02-21T23:53:24.495013+0000","last_peered":"2024-02-21T23:53:24.495013+0000","last_clean":"2024-02-21T23:53:24.495013+0000","last_became_active":"2024-02-21T23:25:17.192394+0000","last_became_peered":"2024-02-21T23:25:17.192394+0000","last_unstale":"2024-02-21T23:53:24.495013+0000","last_undegraded":"2024-02-21T23:53:24.495013+0000","last_fullsized":"2024-02-21T23:53:24.495013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'145","reported_seq":175,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.421556+0000","last_change":"2024-02-21T23:25:21.679491+0000","last_active":"2024-02-21T23:52:56.421556+0000","last_peered":"2024-02-21T23:52:56.421556+0000","last_clean":"2024-02-21T23:52:56.421556+0000","last_became_active":"2024-02-21T23:25:21.679117+0000","last_became_peered":"2024-02-21T23:25:21.679117+0000","last_unstale":"2024-02-21T23:52:56.421556+0000","last_undegraded":"2024-02-21T23:52:56.421556+0000","last_fullsized":"2024-02-21T23:52:56.421556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":9573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"26'38","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626307+0000","last_change":"2024-02-21T23:25:17.161438+0000","last_active":"2024-02-21T23:53:24.626307+0000","last_peered":"2024-02-21T23:53:24.626307+0000","last_clean":"2024-02-21T23:53:24.626307+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:24.626307+0000","last_undegraded":"2024-02-21T23:53:24.626307+0000","last_fullsized":"2024-02-21T23:53:24.626307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:57:38.990056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'648","reported_seq":686,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:07.478231+0000","last_change":"2024-02-21T23:25:16.136913+0000","last_active":"2024-02-21T23:53:07.478231+0000","last_peered":"2024-02-21T23:53:07.478231+0000","last_clean":"2024-02-21T23:53:07.478231+0000","last_became_active":"2024-02-21T23:25:16.136628+0000","last_became_peered":"2024-02-21T23:25:16.136628+0000","last_unstale":"2024-02-21T23:53:07.478231+0000","last_undegraded":"2024-02-21T23:53:07.478231+0000","last_fullsized":"2024-02-21T23:53:07.478231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:11:41.189580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":644,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'134","reported_seq":172,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.695988+0000","last_change":"2024-02-21T23:25:57.526324+0000","last_active":"2024-02-21T23:52:56.695988+0000","last_peered":"2024-02-21T23:52:56.695988+0000","last_clean":"2024-02-21T23:52:56.695988+0000","last_became_active":"2024-02-21T23:25:57.525854+0000","last_became_peered":"2024-02-21T23:25:57.525854+0000","last_unstale":"2024-02-21T23:52:56.695988+0000","last_undegraded":"2024-02-21T23:52:56.695988+0000","last_fullsized":"2024-02-21T23:52:56.695988+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"26'38","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.472520+0000","last_change":"2024-02-21T23:25:17.137829+0000","last_active":"2024-02-21T23:53:24.472520+0000","last_peered":"2024-02-21T23:53:24.472520+0000","last_clean":"2024-02-21T23:53:24.472520+0000","last_became_active":"2024-02-21T23:25:17.137557+0000","last_became_peered":"2024-02-21T23:25:17.137557+0000","last_unstale":"2024-02-21T23:53:24.472520+0000","last_undegraded":"2024-02-21T23:53:24.472520+0000","last_fullsized":"2024-02-21T23:53:24.472520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'2","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:37.473057+0000","last_change":"2024-02-21T23:53:37.473078+0000","last_active":"2024-02-21T23:53:37.473057+0000","last_peered":"2024-02-21T23:53:37.473057+0000","last_clean":"2024-02-21T23:53:37.473057+0000","last_became_active":"2024-02-21T23:25:16.145932+0000","last_became_peered":"2024-02-21T23:25:16.145932+0000","last_unstale":"2024-02-21T23:53:37.473057+0000","last_undegraded":"2024-02-21T23:53:37.473057+0000","last_fullsized":"2024-02-21T23:53:37.473057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-02-21T23:53:37.472982+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-02-21T23:53:37.472982+0000","last_clean_scrub_stamp":"2024-02-21T23:53:37.472982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:07:02.485903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'165","reported_seq":196,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.095880+0000","last_change":"2024-02-21T23:25:21.686044+0000","last_active":"2024-02-21T23:52:55.095880+0000","last_peered":"2024-02-21T23:52:55.095880+0000","last_clean":"2024-02-21T23:52:55.095880+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:52:55.095880+0000","last_undegraded":"2024-02-21T23:52:55.095880+0000","last_fullsized":"2024-02-21T23:52:55.095880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'37","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:28.578496+0000","last_change":"2024-02-21T23:25:17.190236+0000","last_active":"2024-02-21T23:53:28.578496+0000","last_peered":"2024-02-21T23:53:28.578496+0000","last_clean":"2024-02-21T23:53:28.578496+0000","last_became_active":"2024-02-21T23:25:17.189957+0000","last_became_peered":"2024-02-21T23:25:17.189957+0000","last_unstale":"2024-02-21T23:53:28.578496+0000","last_undegraded":"2024-02-21T23:53:28.578496+0000","last_fullsized":"2024-02-21T23:53:28.578496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'2","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:28.476982+0000","last_change":"2024-02-21T23:25:16.148108+0000","last_active":"2024-02-21T23:53:28.476982+0000","last_peered":"2024-02-21T23:53:28.476982+0000","last_clean":"2024-02-21T23:53:28.476982+0000","last_became_active":"2024-02-21T23:25:16.147930+0000","last_became_peered":"2024-02-21T23:25:16.147930+0000","last_unstale":"2024-02-21T23:53:28.476982+0000","last_undegraded":"2024-02-21T23:53:28.476982+0000","last_fullsized":"2024-02-21T23:53:28.476982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'132","reported_seq":163,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.702462+0000","last_change":"2024-02-21T23:25:22.248150+0000","last_active":"2024-02-21T23:52:56.702462+0000","last_peered":"2024-02-21T23:52:56.702462+0000","last_clean":"2024-02-21T23:52:56.702462+0000","last_became_active":"2024-02-21T23:25:22.247769+0000","last_became_peered":"2024-02-21T23:25:22.247769+0000","last_unstale":"2024-02-21T23:52:56.702462+0000","last_undegraded":"2024-02-21T23:52:56.702462+0000","last_fullsized":"2024-02-21T23:52:56.702462+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"26'34","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:28.586605+0000","last_change":"2024-02-21T23:25:17.189758+0000","last_active":"2024-02-21T23:53:28.586605+0000","last_peered":"2024-02-21T23:53:28.586605+0000","last_clean":"2024-02-21T23:53:28.586605+0000","last_became_active":"2024-02-21T23:25:17.189492+0000","last_became_peered":"2024-02-21T23:25:17.189492+0000","last_unstale":"2024-02-21T23:53:28.586605+0000","last_undegraded":"2024-02-21T23:53:28.586605+0000","last_fullsized":"2024-02-21T23:53:28.586605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'7","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:17.858179+0000","last_change":"2024-02-21T23:25:16.131611+0000","last_active":"2024-02-21T23:53:17.858179+0000","last_peered":"2024-02-21T23:53:17.858179+0000","last_clean":"2024-02-21T23:53:17.858179+0000","last_became_active":"2024-02-21T23:25:16.131158+0000","last_became_peered":"2024-02-21T23:25:16.131158+0000","last_unstale":"2024-02-21T23:53:17.858179+0000","last_undegraded":"2024-02-21T23:53:17.858179+0000","last_fullsized":"2024-02-21T23:53:17.858179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'116","reported_seq":147,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.990435+0000","last_change":"2024-02-21T23:25:21.683821+0000","last_active":"2024-02-21T23:52:52.990435+0000","last_peered":"2024-02-21T23:52:52.990435+0000","last_clean":"2024-02-21T23:52:52.990435+0000","last_became_active":"2024-02-21T23:25:21.683466+0000","last_became_peered":"2024-02-21T23:25:21.683466+0000","last_unstale":"2024-02-21T23:52:52.990435+0000","last_undegraded":"2024-02-21T23:52:52.990435+0000","last_fullsized":"2024-02-21T23:52:52.990435+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":7338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"26'38","reported_seq":76,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:28.622207+0000","last_change":"2024-02-21T23:25:17.191394+0000","last_active":"2024-02-21T23:53:28.622207+0000","last_peered":"2024-02-21T23:53:28.622207+0000","last_clean":"2024-02-21T23:53:28.622207+0000","last_became_active":"2024-02-21T23:25:17.191106+0000","last_became_peered":"2024-02-21T23:25:17.191106+0000","last_unstale":"2024-02-21T23:53:28.622207+0000","last_undegraded":"2024-02-21T23:53:28.622207+0000","last_fullsized":"2024-02-21T23:53:28.622207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'9","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.843551+0000","last_change":"2024-02-21T23:25:16.134862+0000","last_active":"2024-02-21T23:53:18.843551+0000","last_peered":"2024-02-21T23:53:18.843551+0000","last_clean":"2024-02-21T23:53:18.843551+0000","last_became_active":"2024-02-21T23:25:16.134602+0000","last_became_peered":"2024-02-21T23:25:16.134602+0000","last_unstale":"2024-02-21T23:53:18.843551+0000","last_undegraded":"2024-02-21T23:53:18.843551+0000","last_fullsized":"2024-02-21T23:53:18.843551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'143","reported_seq":175,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.695933+0000","last_change":"2024-02-21T23:25:22.249606+0000","last_active":"2024-02-21T23:52:56.695933+0000","last_peered":"2024-02-21T23:52:56.695933+0000","last_clean":"2024-02-21T23:52:56.695933+0000","last_became_active":"2024-02-21T23:25:22.249214+0000","last_became_peered":"2024-02-21T23:25:22.249214+0000","last_unstale":"2024-02-21T23:52:56.695933+0000","last_undegraded":"2024-02-21T23:52:56.695933+0000","last_fullsized":"2024-02-21T23:52:56.695933+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":6395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'33","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:28.578566+0000","last_change":"2024-02-21T23:25:17.183290+0000","last_active":"2024-02-21T23:53:28.578566+0000","last_peered":"2024-02-21T23:53:28.578566+0000","last_clean":"2024-02-21T23:53:28.578566+0000","last_became_active":"2024-02-21T23:25:17.183047+0000","last_became_peered":"2024-02-21T23:25:17.183047+0000","last_unstale":"2024-02-21T23:53:28.578566+0000","last_undegraded":"2024-02-21T23:53:28.578566+0000","last_fullsized":"2024-02-21T23:53:28.578566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'1368","reported_seq":1408,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:28.477030+0000","last_change":"2024-02-21T23:25:16.146202+0000","last_active":"2024-02-21T23:53:28.477030+0000","last_peered":"2024-02-21T23:53:28.477030+0000","last_clean":"2024-02-21T23:53:28.477030+0000","last_became_active":"2024-02-21T23:25:16.145911+0000","last_became_peered":"2024-02-21T23:25:16.145911+0000","last_unstale":"2024-02-21T23:53:28.477030+0000","last_undegraded":"2024-02-21T23:53:28.477030+0000","last_fullsized":"2024-02-21T23:53:28.477030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3511,"num_write_kb":1242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'135","reported_seq":168,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.577986+0000","last_change":"2024-02-21T23:25:22.243297+0000","last_active":"2024-02-21T23:53:23.577986+0000","last_peered":"2024-02-21T23:53:23.577986+0000","last_clean":"2024-02-21T23:53:23.577986+0000","last_became_active":"2024-02-21T23:25:22.242922+0000","last_became_peered":"2024-02-21T23:25:22.242922+0000","last_unstale":"2024-02-21T23:53:23.577986+0000","last_undegraded":"2024-02-21T23:53:23.577986+0000","last_fullsized":"2024-02-21T23:53:23.577986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":7447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"26'37","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.466159+0000","last_change":"2024-02-21T23:25:17.190577+0000","last_active":"2024-02-21T23:53:24.466159+0000","last_peered":"2024-02-21T23:53:24.466159+0000","last_clean":"2024-02-21T23:53:24.466159+0000","last_became_active":"2024-02-21T23:25:17.190332+0000","last_became_peered":"2024-02-21T23:25:17.190332+0000","last_unstale":"2024-02-21T23:53:24.466159+0000","last_undegraded":"2024-02-21T23:53:24.466159+0000","last_fullsized":"2024-02-21T23:53:24.466159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'3","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.853192+0000","last_change":"2024-02-21T23:25:16.134076+0000","last_active":"2024-02-21T23:53:15.853192+0000","last_peered":"2024-02-21T23:53:15.853192+0000","last_clean":"2024-02-21T23:53:15.853192+0000","last_became_active":"2024-02-21T23:25:16.133816+0000","last_became_peered":"2024-02-21T23:25:16.133816+0000","last_unstale":"2024-02-21T23:53:15.853192+0000","last_undegraded":"2024-02-21T23:53:15.853192+0000","last_fullsized":"2024-02-21T23:53:15.853192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'127","reported_seq":159,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:52.989478+0000","last_change":"2024-02-21T23:25:22.243360+0000","last_active":"2024-02-21T23:52:52.989478+0000","last_peered":"2024-02-21T23:52:52.989478+0000","last_clean":"2024-02-21T23:52:52.989478+0000","last_became_active":"2024-02-21T23:25:22.242958+0000","last_became_peered":"2024-02-21T23:25:22.242958+0000","last_unstale":"2024-02-21T23:52:52.989478+0000","last_undegraded":"2024-02-21T23:52:52.989478+0000","last_fullsized":"2024-02-21T23:52:52.989478+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":7411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'36","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494810+0000","last_change":"2024-02-21T23:25:17.192402+0000","last_active":"2024-02-21T23:53:24.494810+0000","last_peered":"2024-02-21T23:53:24.494810+0000","last_clean":"2024-02-21T23:53:24.494810+0000","last_became_active":"2024-02-21T23:25:17.192138+0000","last_became_peered":"2024-02-21T23:25:17.192138+0000","last_unstale":"2024-02-21T23:53:24.494810+0000","last_undegraded":"2024-02-21T23:53:24.494810+0000","last_fullsized":"2024-02-21T23:53:24.494810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'2","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:36.444250+0000","last_change":"2024-02-21T23:53:36.444285+0000","last_active":"2024-02-21T23:53:36.444250+0000","last_peered":"2024-02-21T23:53:36.444250+0000","last_clean":"2024-02-21T23:53:36.444250+0000","last_became_active":"2024-02-21T23:25:16.145255+0000","last_became_peered":"2024-02-21T23:25:16.145255+0000","last_unstale":"2024-02-21T23:53:36.444250+0000","last_undegraded":"2024-02-21T23:53:36.444250+0000","last_fullsized":"2024-02-21T23:53:36.444250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-02-21T23:53:36.444180+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-02-21T23:53:36.444180+0000","last_clean_scrub_stamp":"2024-02-21T23:53:36.444180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:13.531450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'208","reported_seq":240,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.702315+0000","last_change":"2024-02-21T23:25:21.683849+0000","last_active":"2024-02-21T23:52:56.702315+0000","last_peered":"2024-02-21T23:52:56.702315+0000","last_clean":"2024-02-21T23:52:56.702315+0000","last_became_active":"2024-02-21T23:25:21.683379+0000","last_became_peered":"2024-02-21T23:25:21.683379+0000","last_unstale":"2024-02-21T23:52:56.702315+0000","last_undegraded":"2024-02-21T23:52:56.702315+0000","last_fullsized":"2024-02-21T23:52:56.702315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":191,"num_write_kb":16487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"26'31","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.485323+0000","last_change":"2024-02-21T23:25:17.166232+0000","last_active":"2024-02-21T23:53:24.485323+0000","last_peered":"2024-02-21T23:53:24.485323+0000","last_clean":"2024-02-21T23:53:24.485323+0000","last_became_active":"2024-02-21T23:25:17.165999+0000","last_became_peered":"2024-02-21T23:25:17.165999+0000","last_unstale":"2024-02-21T23:53:24.485323+0000","last_undegraded":"2024-02-21T23:53:24.485323+0000","last_fullsized":"2024-02-21T23:53:24.485323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:23.691688+0000","last_change":"2024-02-21T23:25:16.147082+0000","last_active":"2024-02-21T23:52:23.691688+0000","last_peered":"2024-02-21T23:52:23.691688+0000","last_clean":"2024-02-21T23:52:23.691688+0000","last_became_active":"2024-02-21T23:25:16.146797+0000","last_became_peered":"2024-02-21T23:25:16.146797+0000","last_unstale":"2024-02-21T23:52:23.691688+0000","last_undegraded":"2024-02-21T23:52:23.691688+0000","last_fullsized":"2024-02-21T23:52:23.691688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'125","reported_seq":151,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:03.619679+0000","last_change":"2024-02-21T23:25:57.583768+0000","last_active":"2024-02-21T23:53:03.619679+0000","last_peered":"2024-02-21T23:53:03.619679+0000","last_clean":"2024-02-21T23:53:03.619679+0000","last_became_active":"2024-02-21T23:25:57.583297+0000","last_became_peered":"2024-02-21T23:25:57.583297+0000","last_unstale":"2024-02-21T23:53:03.619679+0000","last_undegraded":"2024-02-21T23:53:03.619679+0000","last_fullsized":"2024-02-21T23:53:03.619679+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":6113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,6],"acting":[0,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"26'33","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.463077+0000","last_change":"2024-02-21T23:25:17.190425+0000","last_active":"2024-02-21T23:53:24.463077+0000","last_peered":"2024-02-21T23:53:24.463077+0000","last_clean":"2024-02-21T23:53:24.463077+0000","last_became_active":"2024-02-21T23:25:17.190172+0000","last_became_peered":"2024-02-21T23:25:17.190172+0000","last_unstale":"2024-02-21T23:53:24.463077+0000","last_undegraded":"2024-02-21T23:53:24.463077+0000","last_fullsized":"2024-02-21T23:53:24.463077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'999","reported_seq":2041,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.135631+0000","last_change":"2024-02-21T23:53:38.135653+0000","last_active":"2024-02-21T23:53:38.135631+0000","last_peered":"2024-02-21T23:53:38.135631+0000","last_clean":"2024-02-21T23:53:38.135631+0000","last_became_active":"2024-02-21T23:25:16.107822+0000","last_became_peered":"2024-02-21T23:25:16.107822+0000","last_unstale":"2024-02-21T23:53:38.135631+0000","last_undegraded":"2024-02-21T23:53:38.135631+0000","last_fullsized":"2024-02-21T23:53:38.135631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'999","last_scrub_stamp":"2024-02-21T23:53:38.135562+0000","last_deep_scrub":"26'999","last_deep_scrub_stamp":"2024-02-21T23:53:38.135562+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.135562+0000","objects_scrubbed":1,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:04:00.854167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":995,"num_write_kb":1061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'134","reported_seq":166,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:33.477459+0000","last_change":"2024-02-21T23:25:21.686034+0000","last_active":"2024-02-21T23:53:33.477459+0000","last_peered":"2024-02-21T23:53:33.477459+0000","last_clean":"2024-02-21T23:53:33.477459+0000","last_became_active":"2024-02-21T23:25:21.685546+0000","last_became_peered":"2024-02-21T23:25:21.685546+0000","last_unstale":"2024-02-21T23:53:33.477459+0000","last_undegraded":"2024-02-21T23:53:33.477459+0000","last_fullsized":"2024-02-21T23:53:33.477459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":10678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"26'29","reported_seq":67,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607193+0000","last_change":"2024-02-21T23:25:17.161738+0000","last_active":"2024-02-21T23:53:24.607193+0000","last_peered":"2024-02-21T23:53:24.607193+0000","last_clean":"2024-02-21T23:53:24.607193+0000","last_became_active":"2024-02-21T23:25:17.161459+0000","last_became_peered":"2024-02-21T23:25:17.161459+0000","last_unstale":"2024-02-21T23:53:24.607193+0000","last_undegraded":"2024-02-21T23:53:24.607193+0000","last_fullsized":"2024-02-21T23:53:24.607193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'541","reported_seq":581,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:18.846835+0000","last_change":"2024-02-21T23:25:16.147162+0000","last_active":"2024-02-21T23:53:18.846835+0000","last_peered":"2024-02-21T23:53:18.846835+0000","last_clean":"2024-02-21T23:53:18.846835+0000","last_became_active":"2024-02-21T23:25:16.146850+0000","last_became_peered":"2024-02-21T23:25:16.146850+0000","last_unstale":"2024-02-21T23:53:18.846835+0000","last_undegraded":"2024-02-21T23:53:18.846835+0000","last_fullsized":"2024-02-21T23:53:18.846835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'140","reported_seq":172,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:33.622675+0000","last_change":"2024-02-21T23:25:22.248109+0000","last_active":"2024-02-21T23:53:33.622675+0000","last_peered":"2024-02-21T23:53:33.622675+0000","last_clean":"2024-02-21T23:53:33.622675+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:53:33.622675+0000","last_undegraded":"2024-02-21T23:53:33.622675+0000","last_fullsized":"2024-02-21T23:53:33.622675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"26'35","reported_seq":78,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.468334+0000","last_change":"2024-02-21T23:25:57.600968+0000","last_active":"2024-02-21T23:53:24.468334+0000","last_peered":"2024-02-21T23:53:24.468334+0000","last_clean":"2024-02-21T23:53:24.468334+0000","last_became_active":"2024-02-21T23:25:57.600644+0000","last_became_peered":"2024-02-21T23:25:57.600644+0000","last_unstale":"2024-02-21T23:53:24.468334+0000","last_undegraded":"2024-02-21T23:53:24.468334+0000","last_fullsized":"2024-02-21T23:53:24.468334+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'5","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:10.587650+0000","last_change":"2024-02-21T23:25:16.127688+0000","last_active":"2024-02-21T23:53:10.587650+0000","last_peered":"2024-02-21T23:53:10.587650+0000","last_clean":"2024-02-21T23:53:10.587650+0000","last_became_active":"2024-02-21T23:25:16.127388+0000","last_became_peered":"2024-02-21T23:25:16.127388+0000","last_unstale":"2024-02-21T23:53:10.587650+0000","last_undegraded":"2024-02-21T23:53:10.587650+0000","last_fullsized":"2024-02-21T23:53:10.587650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:51:13.500334+0000","last_change":"2024-02-21T23:25:05.612817+0000","last_active":"2024-02-21T23:51:13.500334+0000","last_peered":"2024-02-21T23:51:13.500334+0000","last_clean":"2024-02-21T23:51:13.500334+0000","last_became_active":"2024-02-21T23:25:05.612483+0000","last_became_peered":"2024-02-21T23:25:05.612483+0000","last_unstale":"2024-02-21T23:51:13.500334+0000","last_undegraded":"2024-02-21T23:51:13.500334+0000","last_fullsized":"2024-02-21T23:51:13.500334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:03.736732+0000","last_clean_scrub_stamp":"2024-02-21T23:25:03.736732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:23:33.073040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'168","reported_seq":199,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:55.098108+0000","last_change":"2024-02-21T23:25:22.248174+0000","last_active":"2024-02-21T23:52:55.098108+0000","last_peered":"2024-02-21T23:52:55.098108+0000","last_clean":"2024-02-21T23:52:55.098108+0000","last_became_active":"2024-02-21T23:25:22.247834+0000","last_became_peered":"2024-02-21T23:25:22.247834+0000","last_unstale":"2024-02-21T23:52:55.098108+0000","last_undegraded":"2024-02-21T23:52:55.098108+0000","last_fullsized":"2024-02-21T23:52:55.098108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:13.953358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":10993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604646+0000","last_change":"2024-02-21T23:25:17.192575+0000","last_active":"2024-02-21T23:53:24.604646+0000","last_peered":"2024-02-21T23:53:24.604646+0000","last_clean":"2024-02-21T23:53:24.604646+0000","last_became_active":"2024-02-21T23:25:17.192293+0000","last_became_peered":"2024-02-21T23:25:17.192293+0000","last_unstale":"2024-02-21T23:53:24.604646+0000","last_undegraded":"2024-02-21T23:53:24.604646+0000","last_fullsized":"2024-02-21T23:53:24.604646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:46:39.582659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:11.693433+0000","last_change":"2024-02-21T23:25:16.128369+0000","last_active":"2024-02-21T23:53:11.693433+0000","last_peered":"2024-02-21T23:53:11.693433+0000","last_clean":"2024-02-21T23:53:11.693433+0000","last_became_active":"2024-02-21T23:25:16.128067+0000","last_became_peered":"2024-02-21T23:25:16.128067+0000","last_unstale":"2024-02-21T23:53:11.693433+0000","last_undegraded":"2024-02-21T23:53:11.693433+0000","last_fullsized":"2024-02-21T23:53:11.693433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'138","reported_seq":169,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.761238+0000","last_change":"2024-02-21T23:25:22.243282+0000","last_active":"2024-02-21T23:52:56.761238+0000","last_peered":"2024-02-21T23:52:56.761238+0000","last_clean":"2024-02-21T23:52:56.761238+0000","last_became_active":"2024-02-21T23:25:22.242916+0000","last_became_peered":"2024-02-21T23:25:22.242916+0000","last_unstale":"2024-02-21T23:52:56.761238+0000","last_undegraded":"2024-02-21T23:52:56.761238+0000","last_fullsized":"2024-02-21T23:52:56.761238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:34.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"26'40","reported_seq":76,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493486+0000","last_change":"2024-02-21T23:25:17.185418+0000","last_active":"2024-02-21T23:53:24.493486+0000","last_peered":"2024-02-21T23:53:24.493486+0000","last_clean":"2024-02-21T23:53:24.493486+0000","last_became_active":"2024-02-21T23:25:17.185119+0000","last_became_peered":"2024-02-21T23:25:17.185119+0000","last_unstale":"2024-02-21T23:53:24.493486+0000","last_undegraded":"2024-02-21T23:53:24.493486+0000","last_fullsized":"2024-02-21T23:53:24.493486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'2","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:17.857259+0000","last_change":"2024-02-21T23:25:16.151088+0000","last_active":"2024-02-21T23:53:17.857259+0000","last_peered":"2024-02-21T23:53:17.857259+0000","last_clean":"2024-02-21T23:53:17.857259+0000","last_became_active":"2024-02-21T23:25:16.150819+0000","last_became_peered":"2024-02-21T23:25:16.150819+0000","last_unstale":"2024-02-21T23:53:17.857259+0000","last_undegraded":"2024-02-21T23:53:17.857259+0000","last_fullsized":"2024-02-21T23:53:17.857259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'189","reported_seq":220,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.726673+0000","last_change":"2024-02-21T23:25:21.684592+0000","last_active":"2024-02-21T23:52:56.726673+0000","last_peered":"2024-02-21T23:52:56.726673+0000","last_clean":"2024-02-21T23:52:56.726673+0000","last_became_active":"2024-02-21T23:25:21.684192+0000","last_became_peered":"2024-02-21T23:25:21.684192+0000","last_unstale":"2024-02-21T23:52:56.726673+0000","last_undegraded":"2024-02-21T23:52:56.726673+0000","last_fullsized":"2024-02-21T23:52:56.726673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":12683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.606181+0000","last_change":"2024-02-21T23:25:17.184268+0000","last_active":"2024-02-21T23:53:24.606181+0000","last_peered":"2024-02-21T23:53:24.606181+0000","last_clean":"2024-02-21T23:53:24.606181+0000","last_became_active":"2024-02-21T23:25:17.184027+0000","last_became_peered":"2024-02-21T23:25:17.184027+0000","last_unstale":"2024-02-21T23:53:24.606181+0000","last_undegraded":"2024-02-21T23:53:24.606181+0000","last_fullsized":"2024-02-21T23:53:24.606181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'5","reported_seq":52,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:37.169210+0000","last_change":"2024-02-21T23:53:37.169236+0000","last_active":"2024-02-21T23:53:37.169210+0000","last_peered":"2024-02-21T23:53:37.169210+0000","last_clean":"2024-02-21T23:53:37.169210+0000","last_became_active":"2024-02-21T23:25:16.107815+0000","last_became_peered":"2024-02-21T23:25:16.107815+0000","last_unstale":"2024-02-21T23:53:37.169210+0000","last_undegraded":"2024-02-21T23:53:37.169210+0000","last_fullsized":"2024-02-21T23:53:37.169210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-02-21T23:53:37.169152+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-02-21T23:53:37.169152+0000","last_clean_scrub_stamp":"2024-02-21T23:53:37.169152+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:19:21.138074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'131","reported_seq":162,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.754668+0000","last_change":"2024-02-21T23:25:22.248926+0000","last_active":"2024-02-21T23:52:56.754668+0000","last_peered":"2024-02-21T23:52:56.754668+0000","last_clean":"2024-02-21T23:52:56.754668+0000","last_became_active":"2024-02-21T23:25:22.248558+0000","last_became_peered":"2024-02-21T23:25:22.248558+0000","last_unstale":"2024-02-21T23:52:56.754668+0000","last_undegraded":"2024-02-21T23:52:56.754668+0000","last_fullsized":"2024-02-21T23:52:56.754668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:13:05.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"26'23","reported_seq":59,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605888+0000","last_change":"2024-02-21T23:25:17.183254+0000","last_active":"2024-02-21T23:53:24.605888+0000","last_peered":"2024-02-21T23:53:24.605888+0000","last_clean":"2024-02-21T23:53:24.605888+0000","last_became_active":"2024-02-21T23:25:17.183020+0000","last_became_peered":"2024-02-21T23:25:17.183020+0000","last_unstale":"2024-02-21T23:53:24.605888+0000","last_undegraded":"2024-02-21T23:53:24.605888+0000","last_fullsized":"2024-02-21T23:53:24.605888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'3","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.060277+0000","last_change":"2024-02-21T23:53:38.060301+0000","last_active":"2024-02-21T23:53:38.060277+0000","last_peered":"2024-02-21T23:53:38.060277+0000","last_clean":"2024-02-21T23:53:38.060277+0000","last_became_active":"2024-02-21T23:25:16.128853+0000","last_became_peered":"2024-02-21T23:25:16.128853+0000","last_unstale":"2024-02-21T23:53:38.060277+0000","last_undegraded":"2024-02-21T23:53:38.060277+0000","last_fullsized":"2024-02-21T23:53:38.060277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-02-21T23:53:38.060218+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-02-21T23:53:38.060218+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.060218+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:15.147489+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'163","reported_seq":195,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:33.587006+0000","last_change":"2024-02-21T23:25:21.684342+0000","last_active":"2024-02-21T23:53:33.587006+0000","last_peered":"2024-02-21T23:53:33.587006+0000","last_clean":"2024-02-21T23:53:33.587006+0000","last_became_active":"2024-02-21T23:25:21.683991+0000","last_became_peered":"2024-02-21T23:25:21.683991+0000","last_unstale":"2024-02-21T23:53:33.587006+0000","last_undegraded":"2024-02-21T23:53:33.587006+0000","last_fullsized":"2024-02-21T23:53:33.587006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":12715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"26'37","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.606217+0000","last_change":"2024-02-21T23:25:17.183442+0000","last_active":"2024-02-21T23:53:24.606217+0000","last_peered":"2024-02-21T23:53:24.606217+0000","last_clean":"2024-02-21T23:53:24.606217+0000","last_became_active":"2024-02-21T23:25:17.183057+0000","last_became_peered":"2024-02-21T23:25:17.183057+0000","last_unstale":"2024-02-21T23:53:24.606217+0000","last_undegraded":"2024-02-21T23:53:24.606217+0000","last_fullsized":"2024-02-21T23:53:24.606217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'13","reported_seq":52,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.861300+0000","last_change":"2024-02-21T23:25:16.145801+0000","last_active":"2024-02-21T23:53:23.861300+0000","last_peered":"2024-02-21T23:53:23.861300+0000","last_clean":"2024-02-21T23:53:23.861300+0000","last_became_active":"2024-02-21T23:25:16.145523+0000","last_became_peered":"2024-02-21T23:25:16.145523+0000","last_unstale":"2024-02-21T23:53:23.861300+0000","last_undegraded":"2024-02-21T23:53:23.861300+0000","last_fullsized":"2024-02-21T23:53:23.861300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'157","reported_seq":187,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.687655+0000","last_change":"2024-02-21T23:25:21.679191+0000","last_active":"2024-02-21T23:52:56.687655+0000","last_peered":"2024-02-21T23:52:56.687655+0000","last_clean":"2024-02-21T23:52:56.687655+0000","last_became_active":"2024-02-21T23:25:21.678715+0000","last_became_peered":"2024-02-21T23:25:21.678715+0000","last_unstale":"2024-02-21T23:52:56.687655+0000","last_undegraded":"2024-02-21T23:52:56.687655+0000","last_fullsized":"2024-02-21T23:52:56.687655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":11459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"26'29","reported_seq":65,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.625866+0000","last_change":"2024-02-21T23:25:17.138401+0000","last_active":"2024-02-21T23:53:24.625866+0000","last_peered":"2024-02-21T23:53:24.625866+0000","last_clean":"2024-02-21T23:53:24.625866+0000","last_became_active":"2024-02-21T23:25:17.138136+0000","last_became_peered":"2024-02-21T23:25:17.138136+0000","last_unstale":"2024-02-21T23:53:24.625866+0000","last_undegraded":"2024-02-21T23:53:24.625866+0000","last_fullsized":"2024-02-21T23:53:24.625866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'8","reported_seq":46,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:15.890247+0000","last_change":"2024-02-21T23:25:16.130209+0000","last_active":"2024-02-21T23:53:15.890247+0000","last_peered":"2024-02-21T23:53:15.890247+0000","last_clean":"2024-02-21T23:53:15.890247+0000","last_became_active":"2024-02-21T23:25:16.129974+0000","last_became_peered":"2024-02-21T23:25:16.129974+0000","last_unstale":"2024-02-21T23:53:15.890247+0000","last_undegraded":"2024-02-21T23:53:15.890247+0000","last_fullsized":"2024-02-21T23:53:15.890247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:19:09.094000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'138","reported_seq":168,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.700669+0000","last_change":"2024-02-21T23:25:22.243303+0000","last_active":"2024-02-21T23:52:56.700669+0000","last_peered":"2024-02-21T23:52:56.700669+0000","last_clean":"2024-02-21T23:52:56.700669+0000","last_became_active":"2024-02-21T23:25:22.242960+0000","last_became_peered":"2024-02-21T23:25:22.242960+0000","last_unstale":"2024-02-21T23:52:56.700669+0000","last_undegraded":"2024-02-21T23:52:56.700669+0000","last_fullsized":"2024-02-21T23:52:56.700669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":8319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"26'48","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605915+0000","last_change":"2024-02-21T23:25:17.184233+0000","last_active":"2024-02-21T23:53:24.605915+0000","last_peered":"2024-02-21T23:53:24.605915+0000","last_clean":"2024-02-21T23:53:24.605915+0000","last_became_active":"2024-02-21T23:25:17.183980+0000","last_became_peered":"2024-02-21T23:25:17.183980+0000","last_unstale":"2024-02-21T23:53:24.605915+0000","last_undegraded":"2024-02-21T23:53:24.605915+0000","last_fullsized":"2024-02-21T23:53:24.605915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:26:35.353594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'3","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:14.807121+0000","last_change":"2024-02-21T23:25:16.131147+0000","last_active":"2024-02-21T23:53:14.807121+0000","last_peered":"2024-02-21T23:53:14.807121+0000","last_clean":"2024-02-21T23:53:14.807121+0000","last_became_active":"2024-02-21T23:25:16.130553+0000","last_became_peered":"2024-02-21T23:25:16.130553+0000","last_unstale":"2024-02-21T23:53:14.807121+0000","last_undegraded":"2024-02-21T23:53:14.807121+0000","last_fullsized":"2024-02-21T23:53:14.807121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'97","reported_seq":128,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.578038+0000","last_change":"2024-02-21T23:25:22.243261+0000","last_active":"2024-02-21T23:53:23.578038+0000","last_peered":"2024-02-21T23:53:23.578038+0000","last_clean":"2024-02-21T23:53:23.578038+0000","last_became_active":"2024-02-21T23:25:22.242900+0000","last_became_peered":"2024-02-21T23:25:22.242900+0000","last_unstale":"2024-02-21T23:53:23.578038+0000","last_undegraded":"2024-02-21T23:53:23.578038+0000","last_fullsized":"2024-02-21T23:53:23.578038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":7038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'44","reported_seq":81,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607192+0000","last_change":"2024-02-21T23:25:17.183303+0000","last_active":"2024-02-21T23:53:24.607192+0000","last_peered":"2024-02-21T23:53:24.607192+0000","last_clean":"2024-02-21T23:53:24.607192+0000","last_became_active":"2024-02-21T23:25:17.183042+0000","last_became_peered":"2024-02-21T23:25:17.183042+0000","last_unstale":"2024-02-21T23:53:24.607192+0000","last_undegraded":"2024-02-21T23:53:24.607192+0000","last_fullsized":"2024-02-21T23:53:24.607192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'253","reported_seq":300,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:37.623516+0000","last_change":"2024-02-21T23:53:37.623542+0000","last_active":"2024-02-21T23:53:37.623516+0000","last_peered":"2024-02-21T23:53:37.623516+0000","last_clean":"2024-02-21T23:53:37.623516+0000","last_became_active":"2024-02-21T23:25:16.127375+0000","last_became_peered":"2024-02-21T23:25:16.127375+0000","last_unstale":"2024-02-21T23:53:37.623516+0000","last_undegraded":"2024-02-21T23:53:37.623516+0000","last_fullsized":"2024-02-21T23:53:37.623516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'253","last_scrub_stamp":"2024-02-21T23:53:37.623451+0000","last_deep_scrub":"26'253","last_deep_scrub_stamp":"2024-02-21T23:53:37.623451+0000","last_clean_scrub_stamp":"2024-02-21T23:53:37.623451+0000","objects_scrubbed":3,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:33:06.807958+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2561517,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":2649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'148","reported_seq":177,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.755700+0000","last_change":"2024-02-21T23:25:22.248329+0000","last_active":"2024-02-21T23:52:56.755700+0000","last_peered":"2024-02-21T23:52:56.755700+0000","last_clean":"2024-02-21T23:52:56.755700+0000","last_became_active":"2024-02-21T23:25:22.247889+0000","last_became_peered":"2024-02-21T23:25:22.247889+0000","last_unstale":"2024-02-21T23:52:56.755700+0000","last_undegraded":"2024-02-21T23:52:56.755700+0000","last_fullsized":"2024-02-21T23:52:56.755700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:48:18.308025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":13204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"26'37","reported_seq":59,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.529345+0000","last_change":"2024-02-21T23:25:57.527196+0000","last_active":"2024-02-21T23:53:24.529345+0000","last_peered":"2024-02-21T23:53:24.529345+0000","last_clean":"2024-02-21T23:53:24.529345+0000","last_became_active":"2024-02-21T23:25:57.526874+0000","last_became_peered":"2024-02-21T23:25:57.526874+0000","last_unstale":"2024-02-21T23:53:24.529345+0000","last_undegraded":"2024-02-21T23:53:24.529345+0000","last_fullsized":"2024-02-21T23:53:24.529345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"26'835","reported_seq":873,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:16.019825+0000","last_change":"2024-02-21T23:25:16.116460+0000","last_active":"2024-02-21T23:53:16.019825+0000","last_peered":"2024-02-21T23:53:16.019825+0000","last_clean":"2024-02-21T23:53:16.019825+0000","last_became_active":"2024-02-21T23:25:16.116162+0000","last_became_peered":"2024-02-21T23:25:16.116162+0000","last_unstale":"2024-02-21T23:53:16.019825+0000","last_undegraded":"2024-02-21T23:53:16.019825+0000","last_fullsized":"2024-02-21T23:53:16.019825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:20:41.424169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'155","reported_seq":186,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.476478+0000","last_change":"2024-02-21T23:25:21.683199+0000","last_active":"2024-02-21T23:53:23.476478+0000","last_peered":"2024-02-21T23:53:23.476478+0000","last_clean":"2024-02-21T23:53:23.476478+0000","last_became_active":"2024-02-21T23:25:21.682425+0000","last_became_peered":"2024-02-21T23:25:21.682425+0000","last_unstale":"2024-02-21T23:53:23.476478+0000","last_undegraded":"2024-02-21T23:53:23.476478+0000","last_fullsized":"2024-02-21T23:53:23.476478+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":9421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'32","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.487898+0000","last_change":"2024-02-21T23:25:17.184837+0000","last_active":"2024-02-21T23:53:24.487898+0000","last_peered":"2024-02-21T23:53:24.487898+0000","last_clean":"2024-02-21T23:53:24.487898+0000","last_became_active":"2024-02-21T23:25:17.184574+0000","last_became_peered":"2024-02-21T23:25:17.184574+0000","last_unstale":"2024-02-21T23:53:24.487898+0000","last_undegraded":"2024-02-21T23:53:24.487898+0000","last_fullsized":"2024-02-21T23:53:24.487898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'697","reported_seq":735,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.864715+0000","last_change":"2024-02-21T23:25:16.151031+0000","last_active":"2024-02-21T23:53:23.864715+0000","last_peered":"2024-02-21T23:53:23.864715+0000","last_clean":"2024-02-21T23:53:23.864715+0000","last_became_active":"2024-02-21T23:25:16.150767+0000","last_became_peered":"2024-02-21T23:25:16.150767+0000","last_unstale":"2024-02-21T23:53:23.864715+0000","last_undegraded":"2024-02-21T23:53:23.864715+0000","last_fullsized":"2024-02-21T23:53:23.864715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'152","reported_seq":183,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.586207+0000","last_change":"2024-02-21T23:25:21.683995+0000","last_active":"2024-02-21T23:53:23.586207+0000","last_peered":"2024-02-21T23:53:23.586207+0000","last_clean":"2024-02-21T23:53:23.586207+0000","last_became_active":"2024-02-21T23:25:21.683415+0000","last_became_peered":"2024-02-21T23:25:21.683415+0000","last_unstale":"2024-02-21T23:53:23.586207+0000","last_undegraded":"2024-02-21T23:53:23.586207+0000","last_fullsized":"2024-02-21T23:53:23.586207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"26'42","reported_seq":78,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604776+0000","last_change":"2024-02-21T23:25:17.187230+0000","last_active":"2024-02-21T23:53:24.604776+0000","last_peered":"2024-02-21T23:53:24.604776+0000","last_clean":"2024-02-21T23:53:24.604776+0000","last_became_active":"2024-02-21T23:25:17.186952+0000","last_became_peered":"2024-02-21T23:25:17.186952+0000","last_unstale":"2024-02-21T23:53:24.604776+0000","last_undegraded":"2024-02-21T23:53:24.604776+0000","last_fullsized":"2024-02-21T23:53:24.604776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:00.113126+0000","last_change":"2024-02-21T23:25:16.134722+0000","last_active":"2024-02-21T23:53:00.113126+0000","last_peered":"2024-02-21T23:53:00.113126+0000","last_clean":"2024-02-21T23:53:00.113126+0000","last_became_active":"2024-02-21T23:25:16.134458+0000","last_became_peered":"2024-02-21T23:25:16.134458+0000","last_unstale":"2024-02-21T23:53:00.113126+0000","last_undegraded":"2024-02-21T23:53:00.113126+0000","last_fullsized":"2024-02-21T23:53:00.113126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:06:25.717444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'139","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.578012+0000","last_change":"2024-02-21T23:25:21.684109+0000","last_active":"2024-02-21T23:53:23.578012+0000","last_peered":"2024-02-21T23:53:23.578012+0000","last_clean":"2024-02-21T23:53:23.578012+0000","last_became_active":"2024-02-21T23:25:21.683705+0000","last_became_peered":"2024-02-21T23:25:21.683705+0000","last_unstale":"2024-02-21T23:53:23.578012+0000","last_undegraded":"2024-02-21T23:53:23.578012+0000","last_fullsized":"2024-02-21T23:53:23.578012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":10133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"26'42","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493527+0000","last_change":"2024-02-21T23:25:17.188374+0000","last_active":"2024-02-21T23:53:24.493527+0000","last_peered":"2024-02-21T23:53:24.493527+0000","last_clean":"2024-02-21T23:53:24.493527+0000","last_became_active":"2024-02-21T23:25:17.187792+0000","last_became_peered":"2024-02-21T23:25:17.187792+0000","last_unstale":"2024-02-21T23:53:24.493527+0000","last_undegraded":"2024-02-21T23:53:24.493527+0000","last_fullsized":"2024-02-21T23:53:24.493527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'5","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:01.175408+0000","last_change":"2024-02-21T23:25:16.129446+0000","last_active":"2024-02-21T23:53:01.175408+0000","last_peered":"2024-02-21T23:53:01.175408+0000","last_clean":"2024-02-21T23:53:01.175408+0000","last_became_active":"2024-02-21T23:25:16.129164+0000","last_became_peered":"2024-02-21T23:25:16.129164+0000","last_unstale":"2024-02-21T23:53:01.175408+0000","last_undegraded":"2024-02-21T23:53:01.175408+0000","last_fullsized":"2024-02-21T23:53:01.175408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:09:44.023557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'139","reported_seq":163,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.404867+0000","last_change":"2024-02-21T23:25:57.600307+0000","last_active":"2024-02-21T23:52:56.404867+0000","last_peered":"2024-02-21T23:52:56.404867+0000","last_clean":"2024-02-21T23:52:56.404867+0000","last_became_active":"2024-02-21T23:25:57.599867+0000","last_became_peered":"2024-02-21T23:25:57.599867+0000","last_unstale":"2024-02-21T23:52:56.404867+0000","last_undegraded":"2024-02-21T23:52:56.404867+0000","last_fullsized":"2024-02-21T23:52:56.404867+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":5393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,5],"acting":[2,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"26'153","reported_seq":186,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:23.621487+0000","last_change":"2024-02-21T23:25:22.248119+0000","last_active":"2024-02-21T23:53:23.621487+0000","last_peered":"2024-02-21T23:53:23.621487+0000","last_clean":"2024-02-21T23:53:23.621487+0000","last_became_active":"2024-02-21T23:25:22.247811+0000","last_became_peered":"2024-02-21T23:25:22.247811+0000","last_unstale":"2024-02-21T23:53:23.621487+0000","last_undegraded":"2024-02-21T23:53:23.621487+0000","last_fullsized":"2024-02-21T23:53:23.621487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":11393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'41","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.466354+0000","last_change":"2024-02-21T23:25:17.189176+0000","last_active":"2024-02-21T23:53:24.466354+0000","last_peered":"2024-02-21T23:53:24.466354+0000","last_clean":"2024-02-21T23:53:24.466354+0000","last_became_active":"2024-02-21T23:25:17.188885+0000","last_became_peered":"2024-02-21T23:25:17.188885+0000","last_unstale":"2024-02-21T23:53:24.466354+0000","last_undegraded":"2024-02-21T23:53:24.466354+0000","last_fullsized":"2024-02-21T23:53:24.466354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'4","reported_seq":41,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:04.260921+0000","last_change":"2024-02-21T23:25:16.132036+0000","last_active":"2024-02-21T23:53:04.260921+0000","last_peered":"2024-02-21T23:53:04.260921+0000","last_clean":"2024-02-21T23:53:04.260921+0000","last_became_active":"2024-02-21T23:25:16.131756+0000","last_became_peered":"2024-02-21T23:25:16.131756+0000","last_unstale":"2024-02-21T23:53:04.260921+0000","last_undegraded":"2024-02-21T23:53:04.260921+0000","last_fullsized":"2024-02-21T23:53:04.260921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"26'157","reported_seq":188,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.421511+0000","last_change":"2024-02-21T23:25:21.686133+0000","last_active":"2024-02-21T23:52:56.421511+0000","last_peered":"2024-02-21T23:52:56.421511+0000","last_clean":"2024-02-21T23:52:56.421511+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:52:56.421511+0000","last_undegraded":"2024-02-21T23:52:56.421511+0000","last_fullsized":"2024-02-21T23:52:56.421511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:20.677847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":15074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'44","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471334+0000","last_change":"2024-02-21T23:25:17.135158+0000","last_active":"2024-02-21T23:53:24.471334+0000","last_peered":"2024-02-21T23:53:24.471334+0000","last_clean":"2024-02-21T23:53:24.471334+0000","last_became_active":"2024-02-21T23:25:17.134888+0000","last_became_peered":"2024-02-21T23:25:17.134888+0000","last_unstale":"2024-02-21T23:53:24.471334+0000","last_undegraded":"2024-02-21T23:53:24.471334+0000","last_fullsized":"2024-02-21T23:53:24.471334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'5","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:17.858100+0000","last_change":"2024-02-21T23:25:16.116378+0000","last_active":"2024-02-21T23:53:17.858100+0000","last_peered":"2024-02-21T23:53:17.858100+0000","last_clean":"2024-02-21T23:53:17.858100+0000","last_became_active":"2024-02-21T23:25:16.116201+0000","last_became_peered":"2024-02-21T23:25:16.116201+0000","last_unstale":"2024-02-21T23:53:17.858100+0000","last_undegraded":"2024-02-21T23:53:17.858100+0000","last_fullsized":"2024-02-21T23:53:17.858100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'718","reported_seq":755,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:48.456812+0000","last_change":"2024-02-21T23:25:16.132314+0000","last_active":"2024-02-21T23:52:48.456812+0000","last_peered":"2024-02-21T23:52:48.456812+0000","last_clean":"2024-02-21T23:52:48.456812+0000","last_became_active":"2024-02-21T23:25:16.132062+0000","last_became_peered":"2024-02-21T23:25:16.132062+0000","last_unstale":"2024-02-21T23:52:48.456812+0000","last_undegraded":"2024-02-21T23:52:48.456812+0000","last_fullsized":"2024-02-21T23:52:48.456812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:02:02.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":716,"num_write_kb":4453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'39","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607308+0000","last_change":"2024-02-21T23:25:17.161451+0000","last_active":"2024-02-21T23:53:24.607308+0000","last_peered":"2024-02-21T23:53:24.607308+0000","last_clean":"2024-02-21T23:53:24.607308+0000","last_became_active":"2024-02-21T23:25:17.161176+0000","last_became_peered":"2024-02-21T23:25:17.161176+0000","last_unstale":"2024-02-21T23:53:24.607308+0000","last_undegraded":"2024-02-21T23:53:24.607308+0000","last_fullsized":"2024-02-21T23:53:24.607308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'95","reported_seq":125,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:56.403480+0000","last_change":"2024-02-21T23:25:21.685724+0000","last_active":"2024-02-21T23:52:56.403480+0000","last_peered":"2024-02-21T23:52:56.403480+0000","last_clean":"2024-02-21T23:52:56.403480+0000","last_became_active":"2024-02-21T23:25:21.685375+0000","last_became_peered":"2024-02-21T23:25:21.685375+0000","last_unstale":"2024-02-21T23:52:56.403480+0000","last_undegraded":"2024-02-21T23:52:56.403480+0000","last_fullsized":"2024-02-21T23:52:56.403480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":7997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"26'688","reported_seq":726,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:12.667809+0000","last_change":"2024-02-21T23:25:16.134127+0000","last_active":"2024-02-21T23:53:12.667809+0000","last_peered":"2024-02-21T23:53:12.667809+0000","last_clean":"2024-02-21T23:53:12.667809+0000","last_became_active":"2024-02-21T23:25:16.133839+0000","last_became_peered":"2024-02-21T23:25:16.133839+0000","last_unstale":"2024-02-21T23:53:12.667809+0000","last_undegraded":"2024-02-21T23:53:12.667809+0000","last_fullsized":"2024-02-21T23:53:12.667809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:14.889698+0000","last_clean_scrub_stamp":"2024-02-21T23:25:14.889698+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'33","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.485032+0000","last_change":"2024-02-21T23:25:17.190455+0000","last_active":"2024-02-21T23:53:24.485032+0000","last_peered":"2024-02-21T23:53:24.485032+0000","last_clean":"2024-02-21T23:53:24.485032+0000","last_became_active":"2024-02-21T23:25:17.190194+0000","last_became_peered":"2024-02-21T23:25:17.190194+0000","last_unstale":"2024-02-21T23:53:24.485032+0000","last_undegraded":"2024-02-21T23:53:24.485032+0000","last_fullsized":"2024-02-21T23:53:24.485032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'159","reported_seq":196,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:52:53.575011+0000","last_change":"2024-02-21T23:25:57.526175+0000","last_active":"2024-02-21T23:52:53.575011+0000","last_peered":"2024-02-21T23:52:53.575011+0000","last_clean":"2024-02-21T23:52:53.575011+0000","last_became_active":"2024-02-21T23:25:57.525755+0000","last_became_peered":"2024-02-21T23:25:57.525755+0000","last_unstale":"2024-02-21T23:52:53.575011+0000","last_undegraded":"2024-02-21T23:52:53.575011+0000","last_fullsized":"2024-02-21T23:52:53.575011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:20.401569+0000","last_clean_scrub_stamp":"2024-02-21T23:25:20.401569+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4,2],"acting":[3,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'48","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626470+0000","last_change":"2024-02-21T23:25:17.138039+0000","last_active":"2024-02-21T23:53:24.626470+0000","last_peered":"2024-02-21T23:53:24.626470+0000","last_clean":"2024-02-21T23:53:24.626470+0000","last_became_active":"2024-02-21T23:25:17.137717+0000","last_became_peered":"2024-02-21T23:25:17.137717+0000","last_unstale":"2024-02-21T23:53:24.626470+0000","last_undegraded":"2024-02-21T23:53:24.626470+0000","last_fullsized":"2024-02-21T23:53:24.626470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:29:32.519633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"26'48","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.533181+0000","last_change":"2024-02-21T23:25:17.194727+0000","last_active":"2024-02-21T23:53:24.533181+0000","last_peered":"2024-02-21T23:53:24.533181+0000","last_clean":"2024-02-21T23:53:24.533181+0000","last_became_active":"2024-02-21T23:25:17.194533+0000","last_became_peered":"2024-02-21T23:25:17.194533+0000","last_unstale":"2024-02-21T23:53:24.533181+0000","last_undegraded":"2024-02-21T23:53:24.533181+0000","last_fullsized":"2024-02-21T23:53:24.533181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"26'50","reported_seq":87,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493574+0000","last_change":"2024-02-21T23:25:17.191051+0000","last_active":"2024-02-21T23:53:24.493574+0000","last_peered":"2024-02-21T23:53:24.493574+0000","last_clean":"2024-02-21T23:53:24.493574+0000","last_became_active":"2024-02-21T23:25:17.190600+0000","last_became_peered":"2024-02-21T23:25:17.190600+0000","last_unstale":"2024-02-21T23:53:24.493574+0000","last_undegraded":"2024-02-21T23:53:24.493574+0000","last_fullsized":"2024-02-21T23:53:24.493574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:16:16.268404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"26'36","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.599217+0000","last_change":"2024-02-21T23:25:17.135311+0000","last_active":"2024-02-21T23:53:24.599217+0000","last_peered":"2024-02-21T23:53:24.599217+0000","last_clean":"2024-02-21T23:53:24.599217+0000","last_became_active":"2024-02-21T23:25:17.135011+0000","last_became_peered":"2024-02-21T23:25:17.135011+0000","last_unstale":"2024-02-21T23:53:24.599217+0000","last_undegraded":"2024-02-21T23:53:24.599217+0000","last_fullsized":"2024-02-21T23:53:24.599217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"26'29","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.014008+0000","last_change":"2024-02-21T23:53:38.014030+0000","last_active":"2024-02-21T23:53:38.014008+0000","last_peered":"2024-02-21T23:53:38.014008+0000","last_clean":"2024-02-21T23:53:38.014008+0000","last_became_active":"2024-02-21T23:25:17.161401+0000","last_became_peered":"2024-02-21T23:25:17.161401+0000","last_unstale":"2024-02-21T23:53:38.014008+0000","last_undegraded":"2024-02-21T23:53:38.014008+0000","last_fullsized":"2024-02-21T23:53:38.014008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'29","last_scrub_stamp":"2024-02-21T23:53:38.013931+0000","last_deep_scrub":"26'29","last_deep_scrub_stamp":"2024-02-21T23:53:38.013931+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.013931+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:42.703272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"26'34","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:28.477012+0000","last_change":"2024-02-21T23:25:17.189731+0000","last_active":"2024-02-21T23:53:28.477012+0000","last_peered":"2024-02-21T23:53:28.477012+0000","last_clean":"2024-02-21T23:53:28.477012+0000","last_became_active":"2024-02-21T23:25:17.189484+0000","last_became_peered":"2024-02-21T23:25:17.189484+0000","last_unstale":"2024-02-21T23:53:28.477012+0000","last_undegraded":"2024-02-21T23:53:28.477012+0000","last_fullsized":"2024-02-21T23:53:28.477012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"26'45","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.628365+0000","last_change":"2024-02-21T23:25:17.184125+0000","last_active":"2024-02-21T23:53:24.628365+0000","last_peered":"2024-02-21T23:53:24.628365+0000","last_clean":"2024-02-21T23:53:24.628365+0000","last_became_active":"2024-02-21T23:25:17.183865+0000","last_became_peered":"2024-02-21T23:25:17.183865+0000","last_unstale":"2024-02-21T23:53:24.628365+0000","last_undegraded":"2024-02-21T23:53:24.628365+0000","last_fullsized":"2024-02-21T23:53:24.628365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:18:09.455702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'37","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:28.578555+0000","last_change":"2024-02-21T23:25:17.161482+0000","last_active":"2024-02-21T23:53:28.578555+0000","last_peered":"2024-02-21T23:53:28.578555+0000","last_clean":"2024-02-21T23:53:28.578555+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:28.578555+0000","last_undegraded":"2024-02-21T23:53:28.578555+0000","last_fullsized":"2024-02-21T23:53:28.578555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'32","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494217+0000","last_change":"2024-02-21T23:25:17.185691+0000","last_active":"2024-02-21T23:53:24.494217+0000","last_peered":"2024-02-21T23:53:24.494217+0000","last_clean":"2024-02-21T23:53:24.494217+0000","last_became_active":"2024-02-21T23:25:17.185420+0000","last_became_peered":"2024-02-21T23:25:17.185420+0000","last_unstale":"2024-02-21T23:53:24.494217+0000","last_undegraded":"2024-02-21T23:53:24.494217+0000","last_fullsized":"2024-02-21T23:53:24.494217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"26'46","reported_seq":83,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.471534+0000","last_change":"2024-02-21T23:25:17.161321+0000","last_active":"2024-02-21T23:53:24.471534+0000","last_peered":"2024-02-21T23:53:24.471534+0000","last_clean":"2024-02-21T23:53:24.471534+0000","last_became_active":"2024-02-21T23:25:17.161010+0000","last_became_peered":"2024-02-21T23:25:17.161010+0000","last_unstale":"2024-02-21T23:53:24.471534+0000","last_undegraded":"2024-02-21T23:53:24.471534+0000","last_fullsized":"2024-02-21T23:53:24.471534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"26'39","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494124+0000","last_change":"2024-02-21T23:25:17.191072+0000","last_active":"2024-02-21T23:53:24.494124+0000","last_peered":"2024-02-21T23:53:24.494124+0000","last_clean":"2024-02-21T23:53:24.494124+0000","last_became_active":"2024-02-21T23:25:17.187741+0000","last_became_peered":"2024-02-21T23:25:17.187741+0000","last_unstale":"2024-02-21T23:53:24.494124+0000","last_undegraded":"2024-02-21T23:53:24.494124+0000","last_fullsized":"2024-02-21T23:53:24.494124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:09.543915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"26'53","reported_seq":90,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.607263+0000","last_change":"2024-02-21T23:25:17.186616+0000","last_active":"2024-02-21T23:53:24.607263+0000","last_peered":"2024-02-21T23:53:24.607263+0000","last_clean":"2024-02-21T23:53:24.607263+0000","last_became_active":"2024-02-21T23:25:17.186108+0000","last_became_peered":"2024-02-21T23:25:17.186108+0000","last_unstale":"2024-02-21T23:53:24.607263+0000","last_undegraded":"2024-02-21T23:53:24.607263+0000","last_fullsized":"2024-02-21T23:53:24.607263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"26'43","reported_seq":79,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626297+0000","last_change":"2024-02-21T23:25:17.161428+0000","last_active":"2024-02-21T23:53:24.626297+0000","last_peered":"2024-02-21T23:53:24.626297+0000","last_clean":"2024-02-21T23:53:24.626297+0000","last_became_active":"2024-02-21T23:25:17.161140+0000","last_became_peered":"2024-02-21T23:25:17.161140+0000","last_unstale":"2024-02-21T23:53:24.626297+0000","last_undegraded":"2024-02-21T23:53:24.626297+0000","last_fullsized":"2024-02-21T23:53:24.626297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:59:48.271528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"26'36","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.604720+0000","last_change":"2024-02-21T23:25:17.190666+0000","last_active":"2024-02-21T23:53:24.604720+0000","last_peered":"2024-02-21T23:53:24.604720+0000","last_clean":"2024-02-21T23:53:24.604720+0000","last_became_active":"2024-02-21T23:25:17.190420+0000","last_became_peered":"2024-02-21T23:25:17.190420+0000","last_unstale":"2024-02-21T23:53:24.604720+0000","last_undegraded":"2024-02-21T23:53:24.604720+0000","last_fullsized":"2024-02-21T23:53:24.604720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:19:47.325311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"26'48","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.533185+0000","last_change":"2024-02-21T23:25:17.189583+0000","last_active":"2024-02-21T23:53:24.533185+0000","last_peered":"2024-02-21T23:53:24.533185+0000","last_clean":"2024-02-21T23:53:24.533185+0000","last_became_active":"2024-02-21T23:25:17.189302+0000","last_became_peered":"2024-02-21T23:25:17.189302+0000","last_unstale":"2024-02-21T23:53:24.533185+0000","last_undegraded":"2024-02-21T23:53:24.533185+0000","last_fullsized":"2024-02-21T23:53:24.533185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"26'41","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493492+0000","last_change":"2024-02-21T23:25:17.188113+0000","last_active":"2024-02-21T23:53:24.493492+0000","last_peered":"2024-02-21T23:53:24.493492+0000","last_clean":"2024-02-21T23:53:24.493492+0000","last_became_active":"2024-02-21T23:25:17.187811+0000","last_became_peered":"2024-02-21T23:25:17.187811+0000","last_unstale":"2024-02-21T23:53:24.493492+0000","last_undegraded":"2024-02-21T23:53:24.493492+0000","last_fullsized":"2024-02-21T23:53:24.493492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:11:29.953571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"26'44","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.532363+0000","last_change":"2024-02-21T23:25:57.525755+0000","last_active":"2024-02-21T23:53:24.532363+0000","last_peered":"2024-02-21T23:53:24.532363+0000","last_clean":"2024-02-21T23:53:24.532363+0000","last_became_active":"2024-02-21T23:25:57.525443+0000","last_became_peered":"2024-02-21T23:25:57.525443+0000","last_unstale":"2024-02-21T23:53:24.532363+0000","last_undegraded":"2024-02-21T23:53:24.532363+0000","last_fullsized":"2024-02-21T23:53:24.532363+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"26'38","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.460350+0000","last_change":"2024-02-21T23:25:17.186378+0000","last_active":"2024-02-21T23:53:24.460350+0000","last_peered":"2024-02-21T23:53:24.460350+0000","last_clean":"2024-02-21T23:53:24.460350+0000","last_became_active":"2024-02-21T23:25:17.186086+0000","last_became_peered":"2024-02-21T23:25:17.186086+0000","last_unstale":"2024-02-21T23:53:24.460350+0000","last_undegraded":"2024-02-21T23:53:24.460350+0000","last_fullsized":"2024-02-21T23:53:24.460350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"26'41","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:36.598926+0000","last_change":"2024-02-21T23:53:36.598948+0000","last_active":"2024-02-21T23:53:36.598926+0000","last_peered":"2024-02-21T23:53:36.598926+0000","last_clean":"2024-02-21T23:53:36.598926+0000","last_became_active":"2024-02-21T23:25:17.190680+0000","last_became_peered":"2024-02-21T23:25:17.190680+0000","last_unstale":"2024-02-21T23:53:36.598926+0000","last_undegraded":"2024-02-21T23:53:36.598926+0000","last_fullsized":"2024-02-21T23:53:36.598926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'41","last_scrub_stamp":"2024-02-21T23:53:36.598855+0000","last_deep_scrub":"26'41","last_deep_scrub_stamp":"2024-02-21T23:53:36.598855+0000","last_clean_scrub_stamp":"2024-02-21T23:53:36.598855+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:03:03.021728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"26'38","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.479801+0000","last_change":"2024-02-21T23:25:17.135140+0000","last_active":"2024-02-21T23:53:24.479801+0000","last_peered":"2024-02-21T23:53:24.479801+0000","last_clean":"2024-02-21T23:53:24.479801+0000","last_became_active":"2024-02-21T23:25:17.134866+0000","last_became_peered":"2024-02-21T23:25:17.134866+0000","last_unstale":"2024-02-21T23:53:24.479801+0000","last_undegraded":"2024-02-21T23:53:24.479801+0000","last_fullsized":"2024-02-21T23:53:24.479801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'27","reported_seq":63,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.488861+0000","last_change":"2024-02-21T23:25:17.162204+0000","last_active":"2024-02-21T23:53:24.488861+0000","last_peered":"2024-02-21T23:53:24.488861+0000","last_clean":"2024-02-21T23:53:24.488861+0000","last_became_active":"2024-02-21T23:25:17.161807+0000","last_became_peered":"2024-02-21T23:25:17.161807+0000","last_unstale":"2024-02-21T23:53:24.488861+0000","last_undegraded":"2024-02-21T23:53:24.488861+0000","last_fullsized":"2024-02-21T23:53:24.488861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"26'36","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.626279+0000","last_change":"2024-02-21T23:25:17.188932+0000","last_active":"2024-02-21T23:53:24.626279+0000","last_peered":"2024-02-21T23:53:24.626279+0000","last_clean":"2024-02-21T23:53:24.626279+0000","last_became_active":"2024-02-21T23:25:17.188646+0000","last_became_peered":"2024-02-21T23:25:17.188646+0000","last_unstale":"2024-02-21T23:53:24.626279+0000","last_undegraded":"2024-02-21T23:53:24.626279+0000","last_fullsized":"2024-02-21T23:53:24.626279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:31:12.312537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"26'51","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.628333+0000","last_change":"2024-02-21T23:25:17.188333+0000","last_active":"2024-02-21T23:53:24.628333+0000","last_peered":"2024-02-21T23:53:24.628333+0000","last_clean":"2024-02-21T23:53:24.628333+0000","last_became_active":"2024-02-21T23:25:17.188066+0000","last_became_peered":"2024-02-21T23:25:17.188066+0000","last_unstale":"2024-02-21T23:53:24.628333+0000","last_undegraded":"2024-02-21T23:53:24.628333+0000","last_fullsized":"2024-02-21T23:53:24.628333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:48:57.393725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"26'39","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605416+0000","last_change":"2024-02-21T23:25:17.187549+0000","last_active":"2024-02-21T23:53:24.605416+0000","last_peered":"2024-02-21T23:53:24.605416+0000","last_clean":"2024-02-21T23:53:24.605416+0000","last_became_active":"2024-02-21T23:25:17.187297+0000","last_became_peered":"2024-02-21T23:25:17.187297+0000","last_unstale":"2024-02-21T23:53:24.605416+0000","last_undegraded":"2024-02-21T23:53:24.605416+0000","last_fullsized":"2024-02-21T23:53:24.605416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:29:43.270931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"26'30","reported_seq":52,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:28.624534+0000","last_change":"2024-02-21T23:25:57.526373+0000","last_active":"2024-02-21T23:53:28.624534+0000","last_peered":"2024-02-21T23:53:28.624534+0000","last_clean":"2024-02-21T23:53:28.624534+0000","last_became_active":"2024-02-21T23:25:57.526068+0000","last_became_peered":"2024-02-21T23:25:57.526068+0000","last_unstale":"2024-02-21T23:53:28.624534+0000","last_undegraded":"2024-02-21T23:53:28.624534+0000","last_fullsized":"2024-02-21T23:53:28.624534+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"26'37","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.597841+0000","last_change":"2024-02-21T23:25:17.183264+0000","last_active":"2024-02-21T23:53:24.597841+0000","last_peered":"2024-02-21T23:53:24.597841+0000","last_clean":"2024-02-21T23:53:24.597841+0000","last_became_active":"2024-02-21T23:25:17.182924+0000","last_became_peered":"2024-02-21T23:25:17.182924+0000","last_unstale":"2024-02-21T23:53:24.597841+0000","last_undegraded":"2024-02-21T23:53:24.597841+0000","last_fullsized":"2024-02-21T23:53:24.597841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"26'38","reported_seq":73,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.494000+0000","last_change":"2024-02-21T23:25:17.190841+0000","last_active":"2024-02-21T23:53:24.494000+0000","last_peered":"2024-02-21T23:53:24.494000+0000","last_clean":"2024-02-21T23:53:24.494000+0000","last_became_active":"2024-02-21T23:25:17.190276+0000","last_became_peered":"2024-02-21T23:25:17.190276+0000","last_unstale":"2024-02-21T23:53:24.494000+0000","last_undegraded":"2024-02-21T23:53:24.494000+0000","last_fullsized":"2024-02-21T23:53:24.494000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:20:55.902872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'36","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.601860+0000","last_change":"2024-02-21T23:25:17.189872+0000","last_active":"2024-02-21T23:53:24.601860+0000","last_peered":"2024-02-21T23:53:24.601860+0000","last_clean":"2024-02-21T23:53:24.601860+0000","last_became_active":"2024-02-21T23:25:17.189614+0000","last_became_peered":"2024-02-21T23:25:17.189614+0000","last_unstale":"2024-02-21T23:53:24.601860+0000","last_undegraded":"2024-02-21T23:53:24.601860+0000","last_fullsized":"2024-02-21T23:53:24.601860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:28:01.855941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"26'25","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.493930+0000","last_change":"2024-02-21T23:25:57.600458+0000","last_active":"2024-02-21T23:53:24.493930+0000","last_peered":"2024-02-21T23:53:24.493930+0000","last_clean":"2024-02-21T23:53:24.493930+0000","last_became_active":"2024-02-21T23:25:57.600094+0000","last_became_peered":"2024-02-21T23:25:57.600094+0000","last_unstale":"2024-02-21T23:53:24.493930+0000","last_undegraded":"2024-02-21T23:53:24.493930+0000","last_fullsized":"2024-02-21T23:53:24.493930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:20:16.177299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"26'44","reported_seq":90,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:35.558393+0000","last_change":"2024-02-21T23:53:35.558434+0000","last_active":"2024-02-21T23:53:35.558393+0000","last_peered":"2024-02-21T23:53:35.558393+0000","last_clean":"2024-02-21T23:53:35.558393+0000","last_became_active":"2024-02-21T23:25:17.190576+0000","last_became_peered":"2024-02-21T23:25:17.190576+0000","last_unstale":"2024-02-21T23:53:35.558393+0000","last_undegraded":"2024-02-21T23:53:35.558393+0000","last_fullsized":"2024-02-21T23:53:35.558393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'44","last_scrub_stamp":"2024-02-21T23:53:35.558280+0000","last_deep_scrub":"26'44","last_deep_scrub_stamp":"2024-02-21T23:53:35.558280+0000","last_clean_scrub_stamp":"2024-02-21T23:53:35.558280+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:53:50.422122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"26'34","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.466128+0000","last_change":"2024-02-21T23:25:17.189287+0000","last_active":"2024-02-21T23:53:24.466128+0000","last_peered":"2024-02-21T23:53:24.466128+0000","last_clean":"2024-02-21T23:53:24.466128+0000","last_became_active":"2024-02-21T23:25:17.189029+0000","last_became_peered":"2024-02-21T23:25:17.189029+0000","last_unstale":"2024-02-21T23:53:24.466128+0000","last_undegraded":"2024-02-21T23:53:24.466128+0000","last_fullsized":"2024-02-21T23:53:24.466128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"26'29","reported_seq":66,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:24.605166+0000","last_change":"2024-02-21T23:25:17.183737+0000","last_active":"2024-02-21T23:53:24.605166+0000","last_peered":"2024-02-21T23:53:24.605166+0000","last_clean":"2024-02-21T23:53:24.605166+0000","last_became_active":"2024-02-21T23:25:17.183410+0000","last_became_peered":"2024-02-21T23:25:17.183410+0000","last_unstale":"2024-02-21T23:53:24.605166+0000","last_undegraded":"2024-02-21T23:53:24.605166+0000","last_fullsized":"2024-02-21T23:53:24.605166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-21T23:25:15.992127+0000","last_clean_scrub_stamp":"2024-02-21T23:25:15.992127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:24:35.331012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4087,"num_write_kb":323834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4616,"ondisk_log_size":4616,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2419,"ondisk_log_size":2419,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":40532172,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":11557,"num_write_kb":45747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55631872,"data_stored":81064344,"data_compressed":7543912,"data_compressed_allocated":11722752,"data_compressed_original":37216256,"omap_allocated":0,"internal_metadata":0},"log_size":9537,"ondisk_log_size":9537,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575195,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78552,"kb_used_data":16848,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94293288,"statfs":{"total":96636764160,"available":96556326912,"internally_reserved":0,"allocated":17252352,"data_stored":25871404,"data_compressed":2449552,"data_compressed_allocated":3768320,"data_compressed_original":12558336,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575195,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61012,"kb_used_data":11788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310828,"statfs":{"total":96636764160,"available":96574287872,"internally_reserved":0,"allocated":12070912,"data_stored":16892518,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575195,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":114720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575195,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60916,"kb_used_data":7532,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310924,"statfs":{"total":96636764160,"available":96574386176,"internally_reserved":0,"allocated":7712768,"data_stored":11064847,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575195,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58952,"kb_used_data":9728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312888,"statfs":{"total":96636764160,"available":96576397312,"internally_reserved":0,"allocated":9961472,"data_stored":12918697,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575195,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":495616,"data_stored":335113,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575195,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47824,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324016,"statfs":{"total":96636764160,"available":96587792384,"internally_reserved":0,"allocated":2826240,"data_stored":4309094,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575195,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60600,"kb_used_data":7216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311240,"statfs":{"total":96636764160,"available":96574709760,"internally_reserved":0,"allocated":7389184,"data_stored":11655727,"data_compressed":1158786,"data_compressed_allocated":1720320,"data_compressed_original":6168576,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7045120,"data_stored":10950729,"data_compressed":1126387,"data_compressed_allocated":1675264,"data_compressed_original":5595136,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2547712,"data_stored":4194464,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":220483,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9682944,"data_stored":12804067,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7434240,"data_stored":10950217,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11792384,"data_stored":16777888,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16908288,"data_stored":25166406,"data_compressed":2417153,"data_compressed_allocated":3723264,"data_compressed_original":11984896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-21T23:53:40.925 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.926 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.926 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.926 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.926 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.926 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.926 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.926 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.926 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.926 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.927 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.928 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.928 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.928 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.928 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.928 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.928 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.928 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.928 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.929 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.929 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.929 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.929 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.929 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.929 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.929 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.929 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.929 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.930 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.930 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.930 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.930 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.930 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.930 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.930 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-02-21T23:25:03.736732+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.930 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.930 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.931 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.931 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.931 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.931 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.931 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.931 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.931 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.931 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.931 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.932 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.932 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.932 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.932 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.932 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.932 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.932 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.932 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.932 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.933 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.933 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.933 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.933 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.933 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.933 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.933 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.933 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.933 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.934 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-02-21T23:25:14.889698+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=14, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-02-21T23:25:20.401569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.935 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.936 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.936 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.936 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.936 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.936 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.936 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.936 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.936 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.936 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.937 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.937 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.937 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.937 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.937 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.937 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.937 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.937 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.937 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.938 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.938 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.938 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.938 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.938 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.938 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.938 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.938 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-02-21T23:25:15.992127+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=25, tm_sec=15, tm_wday=2, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=53, tm_sec=33, tm_wday=2, tm_yday=52, tm_isdst=0) 2024-02-21T23:53:40.938 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-02-21T23:54:00.939 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-21T23:54:01.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.056+0000 7fd1fcbdc640 1 Processor -- start 2024-02-21T23:54:01.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.057+0000 7fd1fcbdc640 1 -- start start 2024-02-21T23:54:01.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.057+0000 7fd1fcbdc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80cdfe0 0x7fd1f80c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.057+0000 7fd1fcbdc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f80717e0 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.058+0000 7fd1fcbdc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f8158090 con 0x7fd1f814e510 2024-02-21T23:54:01.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.058+0000 7fd1fcbdc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f80c84b0 con 0x7fd1f80cdfe0 2024-02-21T23:54:01.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.058+0000 7fd1f659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80cdfe0 0x7fd1f80c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.058+0000 7fd1f6d9d640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fd1f814e510 0x7fd1f814e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:37716/0 (socket says 172.21.15.133:37716) 2024-02-21T23:54:01.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.058+0000 7fd1f6d9d640 1 -- 172.21.15.133:0/4095606973 learned_addr learned my addr 172.21.15.133:0/4095606973 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:54:01.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.061+0000 7fd1f659c640 1 -- 172.21.15.133:0/4095606973 >> v1:172.21.15.142:6789/0 conn(0x7fd1f814e510 legacy=0x7fd1f814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.061+0000 7fd1f659c640 1 -- 172.21.15.133:0/4095606973 >> v1:172.21.15.133:6789/0 conn(0x7fd1f80ce3c0 legacy=0x7fd1f8070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:01.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.061+0000 7fd1f659c640 1 -- 172.21.15.133:0/4095606973 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f8158370 con 0x7fd1f80cdfe0 2024-02-21T23:54:01.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.063+0000 7fd1f659c640 1 --2- 172.21.15.133:0/4095606973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80cdfe0 0x7fd1f80c3250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8155b50 tx=0x7fd1ec02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de8dcd9ce55d289 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:01.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.066+0000 7fd1f559a640 1 -- 172.21.15.133:0/4095606973 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec035020 con 0x7fd1f80cdfe0 2024-02-21T23:54:01.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.066+0000 7fd1f559a640 1 -- 172.21.15.133:0/4095606973 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec031800 con 0x7fd1f80cdfe0 2024-02-21T23:54:01.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.066+0000 7fd1f559a640 1 -- 172.21.15.133:0/4095606973 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec031bd0 con 0x7fd1f80cdfe0 2024-02-21T23:54:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.067+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/4095606973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80cdfe0 msgr2=0x7fd1f80c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.067+0000 7fd1fcbdc640 1 --2- 172.21.15.133:0/4095606973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80cdfe0 0x7fd1f80c3250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8155b50 tx=0x7fd1ec02b8a0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.067+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/4095606973 shutdown_connections 2024-02-21T23:54:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.067+0000 7fd1fcbdc640 1 --2- 172.21.15.133:0/4095606973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80cdfe0 0x7fd1f80c3250 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.067+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/4095606973 >> 172.21.15.133:0/4095606973 conn(0x7fd1f806aec0 msgr2=0x7fd1f806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.067+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/4095606973 shutdown_connections 2024-02-21T23:54:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.067+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/4095606973 wait complete. 2024-02-21T23:54:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.068+0000 7fd1fcbdc640 1 Processor -- start 2024-02-21T23:54:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.069+0000 7fd1fcbdc640 1 -- start start 2024-02-21T23:54:01.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.069+0000 7fd1fcbdc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f80cdfe0 0x7fd1f80b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.069+0000 7fd1fcbdc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80ce3c0 0x7fd1f81f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.069+0000 7fd1fcbdc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f814e510 0x7fd1f8152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.069+0000 7fd1fcbdc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8158930 con 0x7fd1f80cdfe0 2024-02-21T23:54:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.069+0000 7fd1fcbdc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8158ca0 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.069+0000 7fd1fcbdc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8071480 con 0x7fd1f814e510 2024-02-21T23:54:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.070+0000 7fd1f5d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80ce3c0 0x7fd1f81f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.070+0000 7fd1f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f80cdfe0 0x7fd1f80b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.070+0000 7fd1f5d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80ce3c0 0x7fd1f81f0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:59774/0 (socket says 172.21.15.133:59774) 2024-02-21T23:54:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.070+0000 7fd1f5d9b640 1 -- 172.21.15.133:0/2423762784 learned_addr learned my addr 172.21.15.133:0/2423762784 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.072+0000 7fd1f6d9d640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f814e510 0x7fd1f8152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.074+0000 7fd1f5d9b640 1 -- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f814e510 msgr2=0x7fd1f8152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.074+0000 7fd1f5d9b640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f814e510 0x7fd1f8152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.074+0000 7fd1f5d9b640 1 -- 172.21.15.133:0/2423762784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f80cdfe0 msgr2=0x7fd1f80b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.074+0000 7fd1f5d9b640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f80cdfe0 0x7fd1f80b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.074+0000 7fd1f5d9b640 1 -- 172.21.15.133:0/2423762784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f80b08b0 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.077+0000 7fd1f5d9b640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80ce3c0 0x7fd1f81f0e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800c410 tx=0x7fd1e800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:01.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.084+0000 7fd1e6ffd640 1 -- 172.21.15.133:0/2423762784 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8017070 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.084+0000 7fd1e6ffd640 1 -- 172.21.15.133:0/2423762784 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e80035f0 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.084+0000 7fd1e6ffd640 1 -- 172.21.15.133:0/2423762784 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8003a20 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.084+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f80b0b20 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.084+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f80b10e0 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.084+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f806a130 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.370+0000 7fd1e6ffd640 1 -- 172.21.15.133:0/2423762784 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1e801a070 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.371+0000 7fd1e6ffd640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd1d806b520 0x7fd1d806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.371+0000 7fd1f659c640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd1d806b520 0x7fd1d806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.372+0000 7fd1f659c640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd1d806b520 0x7fd1d806d9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec036000 tx=0x7fd1ec03c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:01.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.379+0000 7fd1e6ffd640 1 -- 172.21.15.133:0/2423762784 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fd1e8056380 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.390+0000 7fd1e6ffd640 1 -- 172.21.15.133:0/2423762784 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fd1e8021020 con 0x7fd1f80ce3c0 2024-02-21T23:54:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.592+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 --> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1f80caef0 con 0x7fd1d806b520 2024-02-21T23:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.607+0000 7fd1e6ffd640 1 -- 172.21.15.133:0/2423762784 <== mgr.4098 v2:172.21.15.142:6800/3089949271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334484 (secure 0 0 0) 0x7fd1ec09aab0 con 0x7fd1d806b520 2024-02-21T23:54:01.606 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:54:01.608 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-02-21T23:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd1d806b520 msgr2=0x7fd1d806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd1d806b520 0x7fd1d806d9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec036000 tx=0x7fd1ec03c040 comp rx=0 tx=0).stop 2024-02-21T23:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80ce3c0 msgr2=0x7fd1f81f0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80ce3c0 0x7fd1f81f0e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800c410 tx=0x7fd1e800c8d0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 shutdown_connections 2024-02-21T23:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fd1d806b520 0x7fd1d806d9e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f814e510 0x7fd1f8152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80ce3c0 0x7fd1f81f0e40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 --2- 172.21.15.133:0/2423762784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f80cdfe0 0x7fd1f80b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.610+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 >> 172.21.15.133:0/2423762784 conn(0x7fd1f806aec0 msgr2=0x7fd1f8156660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.611+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 shutdown_connections 2024-02-21T23:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.611+0000 7fd1fcbdc640 1 -- 172.21.15.133:0/2423762784 wait complete. 2024-02-21T23:54:01.617 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":888,"stamp":"2024-02-21T23:54:01.048984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":41122540,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":1110,"num_read_kb":1160,"num_write":15803,"num_write_kb":371997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16711,"ondisk_log_size":16711,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410168,"kb_used_data":56632,"kb_used_omap":12,"kb_used_meta":353459,"kb_avail":754564552,"statfs":{"total":773094113280,"available":772674101248,"internally_reserved":0,"allocated":57991168,"data_stored":83162120,"data_compressed":7608710,"data_compressed_allocated":11812864,"data_compressed_original":38363136,"omap_allocated":12750,"internal_metadata":361942578},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002673"},"pg_stats":[{"pgid":"2.1f","version":"26'607","reported_seq":655,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.605301+0000","last_change":"2024-02-21T23:53:38.605353+0000","last_active":"2024-02-21T23:53:38.605301+0000","last_peered":"2024-02-21T23:53:38.605301+0000","last_clean":"2024-02-21T23:53:38.605301+0000","last_became_active":"2024-02-21T23:25:16.134112+0000","last_became_peered":"2024-02-21T23:25:16.134112+0000","last_unstale":"2024-02-21T23:53:38.605301+0000","last_undegraded":"2024-02-21T23:53:38.605301+0000","last_fullsized":"2024-02-21T23:53:38.605301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'607","last_scrub_stamp":"2024-02-21T23:53:38.605182+0000","last_deep_scrub":"26'607","last_deep_scrub_stamp":"2024-02-21T23:53:38.605182+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.605182+0000","objects_scrubbed":3,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:43.294522+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'30","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:45.976322+0000","last_change":"2024-02-21T23:53:45.976345+0000","last_active":"2024-02-21T23:53:45.976322+0000","last_peered":"2024-02-21T23:53:45.976322+0000","last_clean":"2024-02-21T23:53:45.976322+0000","last_became_active":"2024-02-21T23:25:17.135586+0000","last_became_peered":"2024-02-21T23:25:17.135586+0000","last_unstale":"2024-02-21T23:53:45.976322+0000","last_undegraded":"2024-02-21T23:53:45.976322+0000","last_fullsized":"2024-02-21T23:53:45.976322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'30","last_scrub_stamp":"2024-02-21T23:53:45.976246+0000","last_deep_scrub":"26'30","last_deep_scrub_stamp":"2024-02-21T23:53:45.976246+0000","last_clean_scrub_stamp":"2024-02-21T23:53:45.976246+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:43:48.364665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"26'161","reported_seq":199,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:53.845010+0000","last_change":"2024-02-21T23:53:53.845032+0000","last_active":"2024-02-21T23:53:53.845010+0000","last_peered":"2024-02-21T23:53:53.845010+0000","last_clean":"2024-02-21T23:53:53.845010+0000","last_became_active":"2024-02-21T23:25:22.248610+0000","last_became_peered":"2024-02-21T23:25:22.248610+0000","last_unstale":"2024-02-21T23:53:53.845010+0000","last_undegraded":"2024-02-21T23:53:53.845010+0000","last_fullsized":"2024-02-21T23:53:53.845010+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'161","last_scrub_stamp":"2024-02-21T23:53:53.844932+0000","last_deep_scrub":"26'161","last_deep_scrub_stamp":"2024-02-21T23:53:53.844932+0000","last_clean_scrub_stamp":"2024-02-21T23:53:53.844932+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:59:33.706869+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":10997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"26'751","reported_seq":800,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:41.975382+0000","last_change":"2024-02-21T23:53:41.975409+0000","last_active":"2024-02-21T23:53:41.975382+0000","last_peered":"2024-02-21T23:53:41.975382+0000","last_clean":"2024-02-21T23:53:41.975382+0000","last_became_active":"2024-02-21T23:25:16.123374+0000","last_became_peered":"2024-02-21T23:25:16.123374+0000","last_unstale":"2024-02-21T23:53:41.975382+0000","last_undegraded":"2024-02-21T23:53:41.975382+0000","last_fullsized":"2024-02-21T23:53:41.975382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'751","last_scrub_stamp":"2024-02-21T23:53:41.975310+0000","last_deep_scrub":"26'751","last_deep_scrub_stamp":"2024-02-21T23:53:41.975310+0000","last_clean_scrub_stamp":"2024-02-21T23:53:41.975310+0000","objects_scrubbed":1,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:34:15.693263+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":4534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"26'36","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:46.690964+0000","last_change":"2024-02-21T23:53:46.690987+0000","last_active":"2024-02-21T23:53:46.690964+0000","last_peered":"2024-02-21T23:53:46.690964+0000","last_clean":"2024-02-21T23:53:46.690964+0000","last_became_active":"2024-02-21T23:25:17.192626+0000","last_became_peered":"2024-02-21T23:25:17.192626+0000","last_unstale":"2024-02-21T23:53:46.690964+0000","last_undegraded":"2024-02-21T23:53:46.690964+0000","last_fullsized":"2024-02-21T23:53:46.690964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'36","last_scrub_stamp":"2024-02-21T23:53:46.690896+0000","last_deep_scrub":"26'36","last_deep_scrub_stamp":"2024-02-21T23:53:46.690896+0000","last_clean_scrub_stamp":"2024-02-21T23:53:46.690896+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:11.493830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"26'132","reported_seq":177,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:52.060223+0000","last_change":"2024-02-21T23:53:52.060246+0000","last_active":"2024-02-21T23:53:52.060223+0000","last_peered":"2024-02-21T23:53:52.060223+0000","last_clean":"2024-02-21T23:53:52.060223+0000","last_became_active":"2024-02-21T23:25:57.583416+0000","last_became_peered":"2024-02-21T23:25:57.583416+0000","last_unstale":"2024-02-21T23:53:52.060223+0000","last_undegraded":"2024-02-21T23:53:52.060223+0000","last_fullsized":"2024-02-21T23:53:52.060223+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'132","last_scrub_stamp":"2024-02-21T23:53:52.060159+0000","last_deep_scrub":"26'132","last_deep_scrub_stamp":"2024-02-21T23:53:52.060159+0000","last_clean_scrub_stamp":"2024-02-21T23:53:52.060159+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:21:24.345711+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":6267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"26'1346","reported_seq":1392,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:46.078794+0000","last_change":"2024-02-21T23:53:46.078857+0000","last_active":"2024-02-21T23:53:46.078794+0000","last_peered":"2024-02-21T23:53:46.078794+0000","last_clean":"2024-02-21T23:53:46.078794+0000","last_became_active":"2024-02-21T23:25:16.134533+0000","last_became_peered":"2024-02-21T23:25:16.134533+0000","last_unstale":"2024-02-21T23:53:46.078794+0000","last_undegraded":"2024-02-21T23:53:46.078794+0000","last_fullsized":"2024-02-21T23:53:46.078794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1346","last_scrub_stamp":"2024-02-21T23:53:46.078735+0000","last_deep_scrub":"26'1346","last_deep_scrub_stamp":"2024-02-21T23:53:46.078735+0000","last_clean_scrub_stamp":"2024-02-21T23:53:46.078735+0000","objects_scrubbed":3,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:59:51.270039+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1334,"num_write_kb":8974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"26'32","reported_seq":76,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:46.630557+0000","last_change":"2024-02-21T23:53:46.630578+0000","last_active":"2024-02-21T23:53:46.630557+0000","last_peered":"2024-02-21T23:53:46.630557+0000","last_clean":"2024-02-21T23:53:46.630557+0000","last_became_active":"2024-02-21T23:25:17.181886+0000","last_became_peered":"2024-02-21T23:25:17.181886+0000","last_unstale":"2024-02-21T23:53:46.630557+0000","last_undegraded":"2024-02-21T23:53:46.630557+0000","last_fullsized":"2024-02-21T23:53:46.630557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'32","last_scrub_stamp":"2024-02-21T23:53:46.630493+0000","last_deep_scrub":"26'32","last_deep_scrub_stamp":"2024-02-21T23:53:46.630493+0000","last_clean_scrub_stamp":"2024-02-21T23:53:46.630493+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:37:06.887654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"26'141","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:45.055094+0000","last_change":"2024-02-21T23:53:45.055134+0000","last_active":"2024-02-21T23:53:45.055094+0000","last_peered":"2024-02-21T23:53:45.055094+0000","last_clean":"2024-02-21T23:53:45.055094+0000","last_became_active":"2024-02-21T23:25:21.683115+0000","last_became_peered":"2024-02-21T23:25:21.683115+0000","last_unstale":"2024-02-21T23:53:45.055094+0000","last_undegraded":"2024-02-21T23:53:45.055094+0000","last_fullsized":"2024-02-21T23:53:45.055094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'141","last_scrub_stamp":"2024-02-21T23:53:45.054997+0000","last_deep_scrub":"26'141","last_deep_scrub_stamp":"2024-02-21T23:53:45.054997+0000","last_clean_scrub_stamp":"2024-02-21T23:53:45.054997+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:50:05.646374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":12261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"26'7","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:43.021932+0000","last_change":"2024-02-21T23:53:43.021975+0000","last_active":"2024-02-21T23:53:43.021932+0000","last_peered":"2024-02-21T23:53:43.021932+0000","last_clean":"2024-02-21T23:53:43.021932+0000","last_became_active":"2024-02-21T23:25:16.130604+0000","last_became_peered":"2024-02-21T23:25:16.130604+0000","last_unstale":"2024-02-21T23:53:43.021932+0000","last_undegraded":"2024-02-21T23:53:43.021932+0000","last_fullsized":"2024-02-21T23:53:43.021932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7","last_scrub_stamp":"2024-02-21T23:53:43.021858+0000","last_deep_scrub":"26'7","last_deep_scrub_stamp":"2024-02-21T23:53:43.021858+0000","last_clean_scrub_stamp":"2024-02-21T23:53:43.021858+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:33:12.206365+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"26'47","reported_seq":91,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:53.643989+0000","last_change":"2024-02-21T23:53:53.644010+0000","last_active":"2024-02-21T23:53:53.643989+0000","last_peered":"2024-02-21T23:53:53.643989+0000","last_clean":"2024-02-21T23:53:53.643989+0000","last_became_active":"2024-02-21T23:25:17.192394+0000","last_became_peered":"2024-02-21T23:25:17.192394+0000","last_unstale":"2024-02-21T23:53:53.643989+0000","last_undegraded":"2024-02-21T23:53:53.643989+0000","last_fullsized":"2024-02-21T23:53:53.643989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'47","last_scrub_stamp":"2024-02-21T23:53:53.643923+0000","last_deep_scrub":"26'47","last_deep_scrub_stamp":"2024-02-21T23:53:53.643923+0000","last_clean_scrub_stamp":"2024-02-21T23:53:53.643923+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:08:49.849023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"26'145","reported_seq":183,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:44.095617+0000","last_change":"2024-02-21T23:53:44.095663+0000","last_active":"2024-02-21T23:53:44.095617+0000","last_peered":"2024-02-21T23:53:44.095617+0000","last_clean":"2024-02-21T23:53:44.095617+0000","last_became_active":"2024-02-21T23:25:21.679117+0000","last_became_peered":"2024-02-21T23:25:21.679117+0000","last_unstale":"2024-02-21T23:53:44.095617+0000","last_undegraded":"2024-02-21T23:53:44.095617+0000","last_fullsized":"2024-02-21T23:53:44.095617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'145","last_scrub_stamp":"2024-02-21T23:53:44.095511+0000","last_deep_scrub":"26'145","last_deep_scrub_stamp":"2024-02-21T23:53:44.095511+0000","last_clean_scrub_stamp":"2024-02-21T23:53:44.095511+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:07:03.114112+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":9573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"26'38","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:48.337386+0000","last_change":"2024-02-21T23:53:48.337423+0000","last_active":"2024-02-21T23:53:48.337386+0000","last_peered":"2024-02-21T23:53:48.337386+0000","last_clean":"2024-02-21T23:53:48.337386+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:48.337386+0000","last_undegraded":"2024-02-21T23:53:48.337386+0000","last_fullsized":"2024-02-21T23:53:48.337386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'38","last_scrub_stamp":"2024-02-21T23:53:48.337284+0000","last_deep_scrub":"26'38","last_deep_scrub_stamp":"2024-02-21T23:53:48.337284+0000","last_clean_scrub_stamp":"2024-02-21T23:53:48.337284+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:44:57.742175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"26'650","reported_seq":697,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:47.061051+0000","last_change":"2024-02-21T23:53:47.061095+0000","last_active":"2024-02-21T23:53:47.061051+0000","last_peered":"2024-02-21T23:53:47.061051+0000","last_clean":"2024-02-21T23:53:47.061051+0000","last_became_active":"2024-02-21T23:25:16.136628+0000","last_became_peered":"2024-02-21T23:25:16.136628+0000","last_unstale":"2024-02-21T23:53:47.061051+0000","last_undegraded":"2024-02-21T23:53:47.061051+0000","last_fullsized":"2024-02-21T23:53:47.061051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'650","last_scrub_stamp":"2024-02-21T23:53:47.060976+0000","last_deep_scrub":"26'650","last_deep_scrub_stamp":"2024-02-21T23:53:47.060976+0000","last_clean_scrub_stamp":"2024-02-21T23:53:47.060976+0000","objects_scrubbed":2,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:17:28.798162+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":644,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"26'134","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:51.088534+0000","last_change":"2024-02-21T23:53:51.088579+0000","last_active":"2024-02-21T23:53:51.088534+0000","last_peered":"2024-02-21T23:53:51.088534+0000","last_clean":"2024-02-21T23:53:51.088534+0000","last_became_active":"2024-02-21T23:25:57.525854+0000","last_became_peered":"2024-02-21T23:25:57.525854+0000","last_unstale":"2024-02-21T23:53:51.088534+0000","last_undegraded":"2024-02-21T23:53:51.088534+0000","last_fullsized":"2024-02-21T23:53:51.088534+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'134","last_scrub_stamp":"2024-02-21T23:53:51.088461+0000","last_deep_scrub":"26'134","last_deep_scrub_stamp":"2024-02-21T23:53:51.088461+0000","last_clean_scrub_stamp":"2024-02-21T23:53:51.088461+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:25:28.526659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"26'38","reported_seq":83,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:46.989736+0000","last_change":"2024-02-21T23:53:46.989760+0000","last_active":"2024-02-21T23:53:46.989736+0000","last_peered":"2024-02-21T23:53:46.989736+0000","last_clean":"2024-02-21T23:53:46.989736+0000","last_became_active":"2024-02-21T23:25:17.137557+0000","last_became_peered":"2024-02-21T23:25:17.137557+0000","last_unstale":"2024-02-21T23:53:46.989736+0000","last_undegraded":"2024-02-21T23:53:46.989736+0000","last_fullsized":"2024-02-21T23:53:46.989736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'38","last_scrub_stamp":"2024-02-21T23:53:46.989661+0000","last_deep_scrub":"26'38","last_deep_scrub_stamp":"2024-02-21T23:53:46.989661+0000","last_clean_scrub_stamp":"2024-02-21T23:53:46.989661+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:11.792595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"26'2","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:37.473057+0000","last_change":"2024-02-21T23:53:37.473078+0000","last_active":"2024-02-21T23:53:37.473057+0000","last_peered":"2024-02-21T23:53:37.473057+0000","last_clean":"2024-02-21T23:53:37.473057+0000","last_became_active":"2024-02-21T23:25:16.145932+0000","last_became_peered":"2024-02-21T23:25:16.145932+0000","last_unstale":"2024-02-21T23:53:37.473057+0000","last_undegraded":"2024-02-21T23:53:37.473057+0000","last_fullsized":"2024-02-21T23:53:37.473057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-02-21T23:53:37.472982+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-02-21T23:53:37.472982+0000","last_clean_scrub_stamp":"2024-02-21T23:53:37.472982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:07:02.485903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"26'165","reported_seq":204,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:49.116369+0000","last_change":"2024-02-21T23:53:49.116415+0000","last_active":"2024-02-21T23:53:49.116369+0000","last_peered":"2024-02-21T23:53:49.116369+0000","last_clean":"2024-02-21T23:53:49.116369+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:53:49.116369+0000","last_undegraded":"2024-02-21T23:53:49.116369+0000","last_fullsized":"2024-02-21T23:53:49.116369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'165","last_scrub_stamp":"2024-02-21T23:53:49.116265+0000","last_deep_scrub":"26'165","last_deep_scrub_stamp":"2024-02-21T23:53:49.116265+0000","last_clean_scrub_stamp":"2024-02-21T23:53:49.116265+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:52:11.863609+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"26'37","reported_seq":83,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:48.966840+0000","last_change":"2024-02-21T23:53:48.966879+0000","last_active":"2024-02-21T23:53:48.966840+0000","last_peered":"2024-02-21T23:53:48.966840+0000","last_clean":"2024-02-21T23:53:48.966840+0000","last_became_active":"2024-02-21T23:25:17.189957+0000","last_became_peered":"2024-02-21T23:25:17.189957+0000","last_unstale":"2024-02-21T23:53:48.966840+0000","last_undegraded":"2024-02-21T23:53:48.966840+0000","last_fullsized":"2024-02-21T23:53:48.966840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'37","last_scrub_stamp":"2024-02-21T23:53:48.966741+0000","last_deep_scrub":"26'37","last_deep_scrub_stamp":"2024-02-21T23:53:48.966741+0000","last_clean_scrub_stamp":"2024-02-21T23:53:48.966741+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:01:05.842138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"26'2","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:39.537500+0000","last_change":"2024-02-21T23:53:39.537521+0000","last_active":"2024-02-21T23:53:39.537500+0000","last_peered":"2024-02-21T23:53:39.537500+0000","last_clean":"2024-02-21T23:53:39.537500+0000","last_became_active":"2024-02-21T23:25:16.147930+0000","last_became_peered":"2024-02-21T23:25:16.147930+0000","last_unstale":"2024-02-21T23:53:39.537500+0000","last_undegraded":"2024-02-21T23:53:39.537500+0000","last_fullsized":"2024-02-21T23:53:39.537500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-02-21T23:53:39.537437+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-02-21T23:53:39.537437+0000","last_clean_scrub_stamp":"2024-02-21T23:53:39.537437+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:33:08.721943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"26'132","reported_seq":171,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:51.699354+0000","last_change":"2024-02-21T23:53:51.699376+0000","last_active":"2024-02-21T23:53:51.699354+0000","last_peered":"2024-02-21T23:53:51.699354+0000","last_clean":"2024-02-21T23:53:51.699354+0000","last_became_active":"2024-02-21T23:25:22.247769+0000","last_became_peered":"2024-02-21T23:25:22.247769+0000","last_unstale":"2024-02-21T23:53:51.699354+0000","last_undegraded":"2024-02-21T23:53:51.699354+0000","last_fullsized":"2024-02-21T23:53:51.699354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'132","last_scrub_stamp":"2024-02-21T23:53:51.699281+0000","last_deep_scrub":"26'132","last_deep_scrub_stamp":"2024-02-21T23:53:51.699281+0000","last_clean_scrub_stamp":"2024-02-21T23:53:51.699281+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:48:20.573254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"26'34","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:42.071456+0000","last_change":"2024-02-21T23:53:42.071479+0000","last_active":"2024-02-21T23:53:42.071456+0000","last_peered":"2024-02-21T23:53:42.071456+0000","last_clean":"2024-02-21T23:53:42.071456+0000","last_became_active":"2024-02-21T23:25:17.189492+0000","last_became_peered":"2024-02-21T23:25:17.189492+0000","last_unstale":"2024-02-21T23:53:42.071456+0000","last_undegraded":"2024-02-21T23:53:42.071456+0000","last_fullsized":"2024-02-21T23:53:42.071456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'34","last_scrub_stamp":"2024-02-21T23:53:42.071398+0000","last_deep_scrub":"26'34","last_deep_scrub_stamp":"2024-02-21T23:53:42.071398+0000","last_clean_scrub_stamp":"2024-02-21T23:53:42.071398+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:46.760738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"26'7","reported_seq":55,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:44.042372+0000","last_change":"2024-02-21T23:53:44.042394+0000","last_active":"2024-02-21T23:53:44.042372+0000","last_peered":"2024-02-21T23:53:44.042372+0000","last_clean":"2024-02-21T23:53:44.042372+0000","last_became_active":"2024-02-21T23:25:16.131158+0000","last_became_peered":"2024-02-21T23:25:16.131158+0000","last_unstale":"2024-02-21T23:53:44.042372+0000","last_undegraded":"2024-02-21T23:53:44.042372+0000","last_fullsized":"2024-02-21T23:53:44.042372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7","last_scrub_stamp":"2024-02-21T23:53:44.042301+0000","last_deep_scrub":"26'7","last_deep_scrub_stamp":"2024-02-21T23:53:44.042301+0000","last_clean_scrub_stamp":"2024-02-21T23:53:44.042301+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:48.731641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"26'116","reported_seq":155,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:46.080875+0000","last_change":"2024-02-21T23:53:46.080897+0000","last_active":"2024-02-21T23:53:46.080875+0000","last_peered":"2024-02-21T23:53:46.080875+0000","last_clean":"2024-02-21T23:53:46.080875+0000","last_became_active":"2024-02-21T23:25:21.683466+0000","last_became_peered":"2024-02-21T23:25:21.683466+0000","last_unstale":"2024-02-21T23:53:46.080875+0000","last_undegraded":"2024-02-21T23:53:46.080875+0000","last_fullsized":"2024-02-21T23:53:46.080875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'116","last_scrub_stamp":"2024-02-21T23:53:46.080808+0000","last_deep_scrub":"26'116","last_deep_scrub_stamp":"2024-02-21T23:53:46.080808+0000","last_clean_scrub_stamp":"2024-02-21T23:53:46.080808+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:44:55.485699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":7338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"26'38","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:45.691798+0000","last_change":"2024-02-21T23:53:45.691820+0000","last_active":"2024-02-21T23:53:45.691798+0000","last_peered":"2024-02-21T23:53:45.691798+0000","last_clean":"2024-02-21T23:53:45.691798+0000","last_became_active":"2024-02-21T23:25:17.191106+0000","last_became_peered":"2024-02-21T23:25:17.191106+0000","last_unstale":"2024-02-21T23:53:45.691798+0000","last_undegraded":"2024-02-21T23:53:45.691798+0000","last_fullsized":"2024-02-21T23:53:45.691798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'38","last_scrub_stamp":"2024-02-21T23:53:45.691721+0000","last_deep_scrub":"26'38","last_deep_scrub_stamp":"2024-02-21T23:53:45.691721+0000","last_clean_scrub_stamp":"2024-02-21T23:53:45.691721+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:52:08.439065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"26'10","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:44.204800+0000","last_change":"2024-02-21T23:53:44.204843+0000","last_active":"2024-02-21T23:53:44.204800+0000","last_peered":"2024-02-21T23:53:44.204800+0000","last_clean":"2024-02-21T23:53:44.204800+0000","last_became_active":"2024-02-21T23:25:16.134602+0000","last_became_peered":"2024-02-21T23:25:16.134602+0000","last_unstale":"2024-02-21T23:53:44.204800+0000","last_undegraded":"2024-02-21T23:53:44.204800+0000","last_fullsized":"2024-02-21T23:53:44.204800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'10","last_scrub_stamp":"2024-02-21T23:53:44.204728+0000","last_deep_scrub":"26'10","last_deep_scrub_stamp":"2024-02-21T23:53:44.204728+0000","last_clean_scrub_stamp":"2024-02-21T23:53:44.204728+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:48.894068+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"26'143","reported_seq":183,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:50.086945+0000","last_change":"2024-02-21T23:53:50.086969+0000","last_active":"2024-02-21T23:53:50.086945+0000","last_peered":"2024-02-21T23:53:50.086945+0000","last_clean":"2024-02-21T23:53:50.086945+0000","last_became_active":"2024-02-21T23:25:22.249214+0000","last_became_peered":"2024-02-21T23:25:22.249214+0000","last_unstale":"2024-02-21T23:53:50.086945+0000","last_undegraded":"2024-02-21T23:53:50.086945+0000","last_fullsized":"2024-02-21T23:53:50.086945+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'143","last_scrub_stamp":"2024-02-21T23:53:50.086863+0000","last_deep_scrub":"26'143","last_deep_scrub_stamp":"2024-02-21T23:53:50.086863+0000","last_clean_scrub_stamp":"2024-02-21T23:53:50.086863+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:23:15.011167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":6395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'33","reported_seq":79,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.977680+0000","last_change":"2024-02-21T23:53:38.977723+0000","last_active":"2024-02-21T23:53:38.977680+0000","last_peered":"2024-02-21T23:53:38.977680+0000","last_clean":"2024-02-21T23:53:38.977680+0000","last_became_active":"2024-02-21T23:25:17.183047+0000","last_became_peered":"2024-02-21T23:25:17.183047+0000","last_unstale":"2024-02-21T23:53:38.977680+0000","last_undegraded":"2024-02-21T23:53:38.977680+0000","last_fullsized":"2024-02-21T23:53:38.977680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'33","last_scrub_stamp":"2024-02-21T23:53:38.977589+0000","last_deep_scrub":"26'33","last_deep_scrub_stamp":"2024-02-21T23:53:38.977589+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.977589+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:34:00.199970+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'1368","reported_seq":1416,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.511023+0000","last_change":"2024-02-21T23:53:38.511050+0000","last_active":"2024-02-21T23:53:38.511023+0000","last_peered":"2024-02-21T23:53:38.511023+0000","last_clean":"2024-02-21T23:53:38.511023+0000","last_became_active":"2024-02-21T23:25:16.145911+0000","last_became_peered":"2024-02-21T23:25:16.145911+0000","last_unstale":"2024-02-21T23:53:38.511023+0000","last_undegraded":"2024-02-21T23:53:38.511023+0000","last_fullsized":"2024-02-21T23:53:38.511023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1368","last_scrub_stamp":"2024-02-21T23:53:38.510954+0000","last_deep_scrub":"26'1368","last_deep_scrub_stamp":"2024-02-21T23:53:38.510954+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.510954+0000","objects_scrubbed":1,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:03:04.933828+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3511,"num_write_kb":1242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"26'135","reported_seq":176,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:52.873211+0000","last_change":"2024-02-21T23:53:52.873253+0000","last_active":"2024-02-21T23:53:52.873211+0000","last_peered":"2024-02-21T23:53:52.873211+0000","last_clean":"2024-02-21T23:53:52.873211+0000","last_became_active":"2024-02-21T23:25:22.242922+0000","last_became_peered":"2024-02-21T23:25:22.242922+0000","last_unstale":"2024-02-21T23:53:52.873211+0000","last_undegraded":"2024-02-21T23:53:52.873211+0000","last_fullsized":"2024-02-21T23:53:52.873211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'135","last_scrub_stamp":"2024-02-21T23:53:52.873131+0000","last_deep_scrub":"26'135","last_deep_scrub_stamp":"2024-02-21T23:53:52.873131+0000","last_clean_scrub_stamp":"2024-02-21T23:53:52.873131+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:03:12.143711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":7447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"26'37","reported_seq":83,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:47.663071+0000","last_change":"2024-02-21T23:53:47.663096+0000","last_active":"2024-02-21T23:53:47.663071+0000","last_peered":"2024-02-21T23:53:47.663071+0000","last_clean":"2024-02-21T23:53:47.663071+0000","last_became_active":"2024-02-21T23:25:17.190332+0000","last_became_peered":"2024-02-21T23:25:17.190332+0000","last_unstale":"2024-02-21T23:53:47.663071+0000","last_undegraded":"2024-02-21T23:53:47.663071+0000","last_fullsized":"2024-02-21T23:53:47.663071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'37","last_scrub_stamp":"2024-02-21T23:53:47.663000+0000","last_deep_scrub":"26'37","last_deep_scrub_stamp":"2024-02-21T23:53:47.663000+0000","last_clean_scrub_stamp":"2024-02-21T23:53:47.663000+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:43:50.051418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"26'3","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:45.085758+0000","last_change":"2024-02-21T23:53:45.085779+0000","last_active":"2024-02-21T23:53:45.085758+0000","last_peered":"2024-02-21T23:53:45.085758+0000","last_clean":"2024-02-21T23:53:45.085758+0000","last_became_active":"2024-02-21T23:25:16.133816+0000","last_became_peered":"2024-02-21T23:25:16.133816+0000","last_unstale":"2024-02-21T23:53:45.085758+0000","last_undegraded":"2024-02-21T23:53:45.085758+0000","last_fullsized":"2024-02-21T23:53:45.085758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-02-21T23:53:45.085697+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-02-21T23:53:45.085697+0000","last_clean_scrub_stamp":"2024-02-21T23:53:45.085697+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:34:06.308078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"26'127","reported_seq":167,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:49.759533+0000","last_change":"2024-02-21T23:53:49.759557+0000","last_active":"2024-02-21T23:53:49.759533+0000","last_peered":"2024-02-21T23:53:49.759533+0000","last_clean":"2024-02-21T23:53:49.759533+0000","last_became_active":"2024-02-21T23:25:22.242958+0000","last_became_peered":"2024-02-21T23:25:22.242958+0000","last_unstale":"2024-02-21T23:53:49.759533+0000","last_undegraded":"2024-02-21T23:53:49.759533+0000","last_fullsized":"2024-02-21T23:53:49.759533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'127","last_scrub_stamp":"2024-02-21T23:53:49.759452+0000","last_deep_scrub":"26'127","last_deep_scrub_stamp":"2024-02-21T23:53:49.759452+0000","last_clean_scrub_stamp":"2024-02-21T23:53:49.759452+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:48:18.633425+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":7411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"26'36","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:52.621204+0000","last_change":"2024-02-21T23:53:52.621243+0000","last_active":"2024-02-21T23:53:52.621204+0000","last_peered":"2024-02-21T23:53:52.621204+0000","last_clean":"2024-02-21T23:53:52.621204+0000","last_became_active":"2024-02-21T23:25:17.192138+0000","last_became_peered":"2024-02-21T23:25:17.192138+0000","last_unstale":"2024-02-21T23:53:52.621204+0000","last_undegraded":"2024-02-21T23:53:52.621204+0000","last_fullsized":"2024-02-21T23:53:52.621204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'36","last_scrub_stamp":"2024-02-21T23:53:52.621128+0000","last_deep_scrub":"26'36","last_deep_scrub_stamp":"2024-02-21T23:53:52.621128+0000","last_clean_scrub_stamp":"2024-02-21T23:53:52.621128+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:40:34.650115+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"26'2","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:36.444250+0000","last_change":"2024-02-21T23:53:36.444285+0000","last_active":"2024-02-21T23:53:36.444250+0000","last_peered":"2024-02-21T23:53:36.444250+0000","last_clean":"2024-02-21T23:53:36.444250+0000","last_became_active":"2024-02-21T23:25:16.145255+0000","last_became_peered":"2024-02-21T23:25:16.145255+0000","last_unstale":"2024-02-21T23:53:36.444250+0000","last_undegraded":"2024-02-21T23:53:36.444250+0000","last_fullsized":"2024-02-21T23:53:36.444250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-02-21T23:53:36.444180+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-02-21T23:53:36.444180+0000","last_clean_scrub_stamp":"2024-02-21T23:53:36.444180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:13.531450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"26'208","reported_seq":248,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:49.609583+0000","last_change":"2024-02-21T23:53:49.609608+0000","last_active":"2024-02-21T23:53:49.609583+0000","last_peered":"2024-02-21T23:53:49.609583+0000","last_clean":"2024-02-21T23:53:49.609583+0000","last_became_active":"2024-02-21T23:25:21.683379+0000","last_became_peered":"2024-02-21T23:25:21.683379+0000","last_unstale":"2024-02-21T23:53:49.609583+0000","last_undegraded":"2024-02-21T23:53:49.609583+0000","last_fullsized":"2024-02-21T23:53:49.609583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'208","last_scrub_stamp":"2024-02-21T23:53:49.609511+0000","last_deep_scrub":"26'208","last_deep_scrub_stamp":"2024-02-21T23:53:49.609511+0000","last_clean_scrub_stamp":"2024-02-21T23:53:49.609511+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:00:51.621085+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":191,"num_write_kb":16487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"26'31","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:40.072350+0000","last_change":"2024-02-21T23:53:40.072388+0000","last_active":"2024-02-21T23:53:40.072350+0000","last_peered":"2024-02-21T23:53:40.072350+0000","last_clean":"2024-02-21T23:53:40.072350+0000","last_became_active":"2024-02-21T23:25:17.165999+0000","last_became_peered":"2024-02-21T23:25:17.165999+0000","last_unstale":"2024-02-21T23:53:40.072350+0000","last_undegraded":"2024-02-21T23:53:40.072350+0000","last_fullsized":"2024-02-21T23:53:40.072350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'31","last_scrub_stamp":"2024-02-21T23:53:40.072269+0000","last_deep_scrub":"26'31","last_deep_scrub_stamp":"2024-02-21T23:53:40.072269+0000","last_clean_scrub_stamp":"2024-02-21T23:53:40.072269+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:53:40.564178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'2","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:45.632775+0000","last_change":"2024-02-21T23:53:45.632797+0000","last_active":"2024-02-21T23:53:45.632775+0000","last_peered":"2024-02-21T23:53:45.632775+0000","last_clean":"2024-02-21T23:53:45.632775+0000","last_became_active":"2024-02-21T23:25:16.146797+0000","last_became_peered":"2024-02-21T23:25:16.146797+0000","last_unstale":"2024-02-21T23:53:45.632775+0000","last_undegraded":"2024-02-21T23:53:45.632775+0000","last_fullsized":"2024-02-21T23:53:45.632775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-02-21T23:53:45.632714+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-02-21T23:53:45.632714+0000","last_clean_scrub_stamp":"2024-02-21T23:53:45.632714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:44:52.559517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"26'125","reported_seq":159,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:52.837257+0000","last_change":"2024-02-21T23:53:52.837290+0000","last_active":"2024-02-21T23:53:52.837257+0000","last_peered":"2024-02-21T23:53:52.837257+0000","last_clean":"2024-02-21T23:53:52.837257+0000","last_became_active":"2024-02-21T23:25:57.583297+0000","last_became_peered":"2024-02-21T23:25:57.583297+0000","last_unstale":"2024-02-21T23:53:52.837257+0000","last_undegraded":"2024-02-21T23:53:52.837257+0000","last_fullsized":"2024-02-21T23:53:52.837257+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'125","last_scrub_stamp":"2024-02-21T23:53:52.837181+0000","last_deep_scrub":"26'125","last_deep_scrub_stamp":"2024-02-21T23:53:52.837181+0000","last_clean_scrub_stamp":"2024-02-21T23:53:52.837181+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:03:12.107762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":6113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,6],"acting":[0,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"26'33","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:46.615014+0000","last_change":"2024-02-21T23:53:46.615035+0000","last_active":"2024-02-21T23:53:46.615014+0000","last_peered":"2024-02-21T23:53:46.615014+0000","last_clean":"2024-02-21T23:53:46.615014+0000","last_became_active":"2024-02-21T23:25:17.190172+0000","last_became_peered":"2024-02-21T23:25:17.190172+0000","last_unstale":"2024-02-21T23:53:46.615014+0000","last_undegraded":"2024-02-21T23:53:46.615014+0000","last_fullsized":"2024-02-21T23:53:46.615014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'33","last_scrub_stamp":"2024-02-21T23:53:46.614945+0000","last_deep_scrub":"26'33","last_deep_scrub_stamp":"2024-02-21T23:53:46.614945+0000","last_clean_scrub_stamp":"2024-02-21T23:53:46.614945+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:30:35.708900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"26'999","reported_seq":2041,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.135631+0000","last_change":"2024-02-21T23:53:38.135653+0000","last_active":"2024-02-21T23:53:38.135631+0000","last_peered":"2024-02-21T23:53:38.135631+0000","last_clean":"2024-02-21T23:53:38.135631+0000","last_became_active":"2024-02-21T23:25:16.107822+0000","last_became_peered":"2024-02-21T23:25:16.107822+0000","last_unstale":"2024-02-21T23:53:38.135631+0000","last_undegraded":"2024-02-21T23:53:38.135631+0000","last_fullsized":"2024-02-21T23:53:38.135631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'999","last_scrub_stamp":"2024-02-21T23:53:38.135562+0000","last_deep_scrub":"26'999","last_deep_scrub_stamp":"2024-02-21T23:53:38.135562+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.135562+0000","objects_scrubbed":1,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:04:00.854167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":995,"num_write_kb":1061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'134","reported_seq":174,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:50.657141+0000","last_change":"2024-02-21T23:53:50.657164+0000","last_active":"2024-02-21T23:53:50.657141+0000","last_peered":"2024-02-21T23:53:50.657141+0000","last_clean":"2024-02-21T23:53:50.657141+0000","last_became_active":"2024-02-21T23:25:21.685546+0000","last_became_peered":"2024-02-21T23:25:21.685546+0000","last_unstale":"2024-02-21T23:53:50.657141+0000","last_undegraded":"2024-02-21T23:53:50.657141+0000","last_fullsized":"2024-02-21T23:53:50.657141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'134","last_scrub_stamp":"2024-02-21T23:53:50.657059+0000","last_deep_scrub":"26'134","last_deep_scrub_stamp":"2024-02-21T23:53:50.657059+0000","last_clean_scrub_stamp":"2024-02-21T23:53:50.657059+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:38:53.108179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":10678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"26'29","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:41.646824+0000","last_change":"2024-02-21T23:53:41.646865+0000","last_active":"2024-02-21T23:53:41.646824+0000","last_peered":"2024-02-21T23:53:41.646824+0000","last_clean":"2024-02-21T23:53:41.646824+0000","last_became_active":"2024-02-21T23:25:17.161459+0000","last_became_peered":"2024-02-21T23:25:17.161459+0000","last_unstale":"2024-02-21T23:53:41.646824+0000","last_undegraded":"2024-02-21T23:53:41.646824+0000","last_fullsized":"2024-02-21T23:53:41.646824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'29","last_scrub_stamp":"2024-02-21T23:53:41.646731+0000","last_deep_scrub":"26'29","last_deep_scrub_stamp":"2024-02-21T23:53:41.646731+0000","last_clean_scrub_stamp":"2024-02-21T23:53:41.646731+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:17:23.383917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"26'542","reported_seq":591,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:48.080169+0000","last_change":"2024-02-21T23:53:48.080210+0000","last_active":"2024-02-21T23:53:48.080169+0000","last_peered":"2024-02-21T23:53:48.080169+0000","last_clean":"2024-02-21T23:53:48.080169+0000","last_became_active":"2024-02-21T23:25:16.146850+0000","last_became_peered":"2024-02-21T23:25:16.146850+0000","last_unstale":"2024-02-21T23:53:48.080169+0000","last_undegraded":"2024-02-21T23:53:48.080169+0000","last_fullsized":"2024-02-21T23:53:48.080169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'542","last_scrub_stamp":"2024-02-21T23:53:48.080075+0000","last_deep_scrub":"26'542","last_deep_scrub_stamp":"2024-02-21T23:53:48.080075+0000","last_clean_scrub_stamp":"2024-02-21T23:53:48.080075+0000","objects_scrubbed":1,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:02:08.782578+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"26'140","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:50.808635+0000","last_change":"2024-02-21T23:53:50.808677+0000","last_active":"2024-02-21T23:53:50.808635+0000","last_peered":"2024-02-21T23:53:50.808635+0000","last_clean":"2024-02-21T23:53:50.808635+0000","last_became_active":"2024-02-21T23:25:22.247755+0000","last_became_peered":"2024-02-21T23:25:22.247755+0000","last_unstale":"2024-02-21T23:53:50.808635+0000","last_undegraded":"2024-02-21T23:53:50.808635+0000","last_fullsized":"2024-02-21T23:53:50.808635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'140","last_scrub_stamp":"2024-02-21T23:53:50.808528+0000","last_deep_scrub":"26'140","last_deep_scrub_stamp":"2024-02-21T23:53:50.808528+0000","last_clean_scrub_stamp":"2024-02-21T23:53:50.808528+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:08:47.013629+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"26'35","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:55.881263+0000","last_change":"2024-02-21T23:53:55.881311+0000","last_active":"2024-02-21T23:53:55.881263+0000","last_peered":"2024-02-21T23:53:55.881263+0000","last_clean":"2024-02-21T23:53:55.881263+0000","last_became_active":"2024-02-21T23:25:57.600644+0000","last_became_peered":"2024-02-21T23:25:57.600644+0000","last_unstale":"2024-02-21T23:53:55.881263+0000","last_undegraded":"2024-02-21T23:53:55.881263+0000","last_fullsized":"2024-02-21T23:53:55.881263+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'35","last_scrub_stamp":"2024-02-21T23:53:55.881166+0000","last_deep_scrub":"26'35","last_deep_scrub_stamp":"2024-02-21T23:53:55.881166+0000","last_clean_scrub_stamp":"2024-02-21T23:53:55.881166+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:53:27.483975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'5","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:41.723283+0000","last_change":"2024-02-21T23:53:41.723303+0000","last_active":"2024-02-21T23:53:41.723283+0000","last_peered":"2024-02-21T23:53:41.723283+0000","last_clean":"2024-02-21T23:53:41.723283+0000","last_became_active":"2024-02-21T23:25:16.127388+0000","last_became_peered":"2024-02-21T23:25:16.127388+0000","last_unstale":"2024-02-21T23:53:41.723283+0000","last_undegraded":"2024-02-21T23:53:41.723283+0000","last_fullsized":"2024-02-21T23:53:41.723283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-02-21T23:53:41.723214+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-02-21T23:53:41.723214+0000","last_clean_scrub_stamp":"2024-02-21T23:53:41.723214+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:34:02.945595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"26'119","reported_seq":207,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:42.050654+0000","last_change":"2024-02-21T23:53:42.050712+0000","last_active":"2024-02-21T23:53:42.050654+0000","last_peered":"2024-02-21T23:53:42.050654+0000","last_clean":"2024-02-21T23:53:42.050654+0000","last_became_active":"2024-02-21T23:25:05.612483+0000","last_became_peered":"2024-02-21T23:25:05.612483+0000","last_unstale":"2024-02-21T23:53:42.050654+0000","last_undegraded":"2024-02-21T23:53:42.050654+0000","last_fullsized":"2024-02-21T23:53:42.050654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'119","last_scrub_stamp":"2024-02-21T23:53:42.050577+0000","last_deep_scrub":"26'119","last_deep_scrub_stamp":"2024-02-21T23:53:42.050577+0000","last_clean_scrub_stamp":"2024-02-21T23:53:42.050577+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:03:08.473451+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"26'168","reported_seq":208,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:54.398237+0000","last_change":"2024-02-21T23:53:54.398259+0000","last_active":"2024-02-21T23:53:54.398237+0000","last_peered":"2024-02-21T23:53:54.398237+0000","last_clean":"2024-02-21T23:53:54.398237+0000","last_became_active":"2024-02-21T23:25:22.247834+0000","last_became_peered":"2024-02-21T23:25:22.247834+0000","last_unstale":"2024-02-21T23:53:54.398237+0000","last_undegraded":"2024-02-21T23:53:54.398237+0000","last_fullsized":"2024-02-21T23:53:54.398237+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'168","last_scrub_stamp":"2024-02-21T23:53:54.398167+0000","last_deep_scrub":"26'168","last_deep_scrub_stamp":"2024-02-21T23:53:54.398167+0000","last_clean_scrub_stamp":"2024-02-21T23:53:54.398167+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:00:56.409742+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":10993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"26'36","reported_seq":81,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:56.931503+0000","last_change":"2024-02-21T23:53:56.931550+0000","last_active":"2024-02-21T23:53:56.931503+0000","last_peered":"2024-02-21T23:53:56.931503+0000","last_clean":"2024-02-21T23:53:56.931503+0000","last_became_active":"2024-02-21T23:25:17.192293+0000","last_became_peered":"2024-02-21T23:25:17.192293+0000","last_unstale":"2024-02-21T23:53:56.931503+0000","last_undegraded":"2024-02-21T23:53:56.931503+0000","last_fullsized":"2024-02-21T23:53:56.931503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'36","last_scrub_stamp":"2024-02-21T23:53:56.931424+0000","last_deep_scrub":"26'36","last_deep_scrub_stamp":"2024-02-21T23:53:56.931424+0000","last_clean_scrub_stamp":"2024-02-21T23:53:56.931424+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:08:53.136524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"26'2","reported_seq":49,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:39.172159+0000","last_change":"2024-02-21T23:53:39.172181+0000","last_active":"2024-02-21T23:53:39.172159+0000","last_peered":"2024-02-21T23:53:39.172159+0000","last_clean":"2024-02-21T23:53:39.172159+0000","last_became_active":"2024-02-21T23:25:16.128067+0000","last_became_peered":"2024-02-21T23:25:16.128067+0000","last_unstale":"2024-02-21T23:53:39.172159+0000","last_undegraded":"2024-02-21T23:53:39.172159+0000","last_fullsized":"2024-02-21T23:53:39.172159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-02-21T23:53:39.172088+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-02-21T23:53:39.172088+0000","last_clean_scrub_stamp":"2024-02-21T23:53:39.172088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:37:52.298665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"26'138","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:53.427293+0000","last_change":"2024-02-21T23:53:53.427315+0000","last_active":"2024-02-21T23:53:53.427293+0000","last_peered":"2024-02-21T23:53:53.427293+0000","last_clean":"2024-02-21T23:53:53.427293+0000","last_became_active":"2024-02-21T23:25:22.242916+0000","last_became_peered":"2024-02-21T23:25:22.242916+0000","last_unstale":"2024-02-21T23:53:53.427293+0000","last_undegraded":"2024-02-21T23:53:53.427293+0000","last_fullsized":"2024-02-21T23:53:53.427293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'138","last_scrub_stamp":"2024-02-21T23:53:53.427223+0000","last_deep_scrub":"26'138","last_deep_scrub_stamp":"2024-02-21T23:53:53.427223+0000","last_clean_scrub_stamp":"2024-02-21T23:53:53.427223+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:18.230157+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":6185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"26'40","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:47.584338+0000","last_change":"2024-02-21T23:53:47.584359+0000","last_active":"2024-02-21T23:53:47.584338+0000","last_peered":"2024-02-21T23:53:47.584338+0000","last_clean":"2024-02-21T23:53:47.584338+0000","last_became_active":"2024-02-21T23:25:17.185119+0000","last_became_peered":"2024-02-21T23:25:17.185119+0000","last_unstale":"2024-02-21T23:53:47.584338+0000","last_undegraded":"2024-02-21T23:53:47.584338+0000","last_fullsized":"2024-02-21T23:53:47.584338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'40","last_scrub_stamp":"2024-02-21T23:53:47.584271+0000","last_deep_scrub":"26'40","last_deep_scrub_stamp":"2024-02-21T23:53:47.584271+0000","last_clean_scrub_stamp":"2024-02-21T23:53:47.584271+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:30:36.678226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"26'3","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:41.553387+0000","last_change":"2024-02-21T23:53:41.553410+0000","last_active":"2024-02-21T23:53:41.553387+0000","last_peered":"2024-02-21T23:53:41.553387+0000","last_clean":"2024-02-21T23:53:41.553387+0000","last_became_active":"2024-02-21T23:25:16.150819+0000","last_became_peered":"2024-02-21T23:25:16.150819+0000","last_unstale":"2024-02-21T23:53:41.553387+0000","last_undegraded":"2024-02-21T23:53:41.553387+0000","last_fullsized":"2024-02-21T23:53:41.553387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-02-21T23:53:41.553322+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-02-21T23:53:41.553322+0000","last_clean_scrub_stamp":"2024-02-21T23:53:41.553322+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:28:14.105554+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"26'189","reported_seq":228,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:48.738830+0000","last_change":"2024-02-21T23:53:48.738853+0000","last_active":"2024-02-21T23:53:48.738830+0000","last_peered":"2024-02-21T23:53:48.738830+0000","last_clean":"2024-02-21T23:53:48.738830+0000","last_became_active":"2024-02-21T23:25:21.684192+0000","last_became_peered":"2024-02-21T23:25:21.684192+0000","last_unstale":"2024-02-21T23:53:48.738830+0000","last_undegraded":"2024-02-21T23:53:48.738830+0000","last_fullsized":"2024-02-21T23:53:48.738830+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'189","last_scrub_stamp":"2024-02-21T23:53:48.738763+0000","last_deep_scrub":"26'189","last_deep_scrub_stamp":"2024-02-21T23:53:48.738763+0000","last_clean_scrub_stamp":"2024-02-21T23:53:48.738763+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:01:05.614159+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":12683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"26'36","reported_seq":81,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:52.991691+0000","last_change":"2024-02-21T23:53:52.991736+0000","last_active":"2024-02-21T23:53:52.991691+0000","last_peered":"2024-02-21T23:53:52.991691+0000","last_clean":"2024-02-21T23:53:52.991691+0000","last_became_active":"2024-02-21T23:25:17.184027+0000","last_became_peered":"2024-02-21T23:25:17.184027+0000","last_unstale":"2024-02-21T23:53:52.991691+0000","last_undegraded":"2024-02-21T23:53:52.991691+0000","last_fullsized":"2024-02-21T23:53:52.991691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'36","last_scrub_stamp":"2024-02-21T23:53:52.991569+0000","last_deep_scrub":"26'36","last_deep_scrub_stamp":"2024-02-21T23:53:52.991569+0000","last_clean_scrub_stamp":"2024-02-21T23:53:52.991569+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:25:30.429767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"26'5","reported_seq":52,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:37.169210+0000","last_change":"2024-02-21T23:53:37.169236+0000","last_active":"2024-02-21T23:53:37.169210+0000","last_peered":"2024-02-21T23:53:37.169210+0000","last_clean":"2024-02-21T23:53:37.169210+0000","last_became_active":"2024-02-21T23:25:16.107815+0000","last_became_peered":"2024-02-21T23:25:16.107815+0000","last_unstale":"2024-02-21T23:53:37.169210+0000","last_undegraded":"2024-02-21T23:53:37.169210+0000","last_fullsized":"2024-02-21T23:53:37.169210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-02-21T23:53:37.169152+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-02-21T23:53:37.169152+0000","last_clean_scrub_stamp":"2024-02-21T23:53:37.169152+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:19:21.138074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"26'131","reported_seq":171,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:55.604992+0000","last_change":"2024-02-21T23:53:55.605038+0000","last_active":"2024-02-21T23:53:55.604992+0000","last_peered":"2024-02-21T23:53:55.604992+0000","last_clean":"2024-02-21T23:53:55.604992+0000","last_became_active":"2024-02-21T23:25:22.248558+0000","last_became_peered":"2024-02-21T23:25:22.248558+0000","last_unstale":"2024-02-21T23:53:55.604992+0000","last_undegraded":"2024-02-21T23:53:55.604992+0000","last_fullsized":"2024-02-21T23:53:55.604992+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'131","last_scrub_stamp":"2024-02-21T23:53:55.604898+0000","last_deep_scrub":"26'131","last_deep_scrub_stamp":"2024-02-21T23:53:55.604898+0000","last_clean_scrub_stamp":"2024-02-21T23:53:55.604898+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:03:14.875478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"26'23","reported_seq":68,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:41.015185+0000","last_change":"2024-02-21T23:53:41.015208+0000","last_active":"2024-02-21T23:53:41.015185+0000","last_peered":"2024-02-21T23:53:41.015185+0000","last_clean":"2024-02-21T23:53:41.015185+0000","last_became_active":"2024-02-21T23:25:17.183020+0000","last_became_peered":"2024-02-21T23:25:17.183020+0000","last_unstale":"2024-02-21T23:53:41.015185+0000","last_undegraded":"2024-02-21T23:53:41.015185+0000","last_fullsized":"2024-02-21T23:53:41.015185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'23","last_scrub_stamp":"2024-02-21T23:53:41.015108+0000","last_deep_scrub":"26'23","last_deep_scrub_stamp":"2024-02-21T23:53:41.015108+0000","last_clean_scrub_stamp":"2024-02-21T23:53:41.015108+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:53:55.878950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"26'3","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.060277+0000","last_change":"2024-02-21T23:53:38.060301+0000","last_active":"2024-02-21T23:53:38.060277+0000","last_peered":"2024-02-21T23:53:38.060277+0000","last_clean":"2024-02-21T23:53:38.060277+0000","last_became_active":"2024-02-21T23:25:16.128853+0000","last_became_peered":"2024-02-21T23:25:16.128853+0000","last_unstale":"2024-02-21T23:53:38.060277+0000","last_undegraded":"2024-02-21T23:53:38.060277+0000","last_fullsized":"2024-02-21T23:53:38.060277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-02-21T23:53:38.060218+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-02-21T23:53:38.060218+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.060218+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:31:15.147489+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"26'163","reported_seq":203,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:48.069187+0000","last_change":"2024-02-21T23:53:48.069230+0000","last_active":"2024-02-21T23:53:48.069187+0000","last_peered":"2024-02-21T23:53:48.069187+0000","last_clean":"2024-02-21T23:53:48.069187+0000","last_became_active":"2024-02-21T23:25:21.683991+0000","last_became_peered":"2024-02-21T23:25:21.683991+0000","last_unstale":"2024-02-21T23:53:48.069187+0000","last_undegraded":"2024-02-21T23:53:48.069187+0000","last_fullsized":"2024-02-21T23:53:48.069187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'163","last_scrub_stamp":"2024-02-21T23:53:48.069101+0000","last_deep_scrub":"26'163","last_deep_scrub_stamp":"2024-02-21T23:53:48.069101+0000","last_clean_scrub_stamp":"2024-02-21T23:53:48.069101+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:37:08.326262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":12715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"26'37","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:50.095700+0000","last_change":"2024-02-21T23:53:50.095720+0000","last_active":"2024-02-21T23:53:50.095700+0000","last_peered":"2024-02-21T23:53:50.095700+0000","last_clean":"2024-02-21T23:53:50.095700+0000","last_became_active":"2024-02-21T23:25:17.183057+0000","last_became_peered":"2024-02-21T23:25:17.183057+0000","last_unstale":"2024-02-21T23:53:50.095700+0000","last_undegraded":"2024-02-21T23:53:50.095700+0000","last_fullsized":"2024-02-21T23:53:50.095700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'37","last_scrub_stamp":"2024-02-21T23:53:50.095631+0000","last_deep_scrub":"26'37","last_deep_scrub_stamp":"2024-02-21T23:53:50.095631+0000","last_clean_scrub_stamp":"2024-02-21T23:53:50.095631+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:37:10.352792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"26'17","reported_seq":64,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:40.182743+0000","last_change":"2024-02-21T23:53:40.182793+0000","last_active":"2024-02-21T23:53:40.182743+0000","last_peered":"2024-02-21T23:53:40.182743+0000","last_clean":"2024-02-21T23:53:40.182743+0000","last_became_active":"2024-02-21T23:25:16.145523+0000","last_became_peered":"2024-02-21T23:25:16.145523+0000","last_unstale":"2024-02-21T23:53:40.182743+0000","last_undegraded":"2024-02-21T23:53:40.182743+0000","last_fullsized":"2024-02-21T23:53:40.182743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'17","last_scrub_stamp":"2024-02-21T23:53:40.182646+0000","last_deep_scrub":"26'17","last_deep_scrub_stamp":"2024-02-21T23:53:40.182646+0000","last_clean_scrub_stamp":"2024-02-21T23:53:40.182646+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:04:13.391496+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"26'157","reported_seq":195,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:47.729386+0000","last_change":"2024-02-21T23:53:47.729411+0000","last_active":"2024-02-21T23:53:47.729386+0000","last_peered":"2024-02-21T23:53:47.729386+0000","last_clean":"2024-02-21T23:53:47.729386+0000","last_became_active":"2024-02-21T23:25:21.678715+0000","last_became_peered":"2024-02-21T23:25:21.678715+0000","last_unstale":"2024-02-21T23:53:47.729386+0000","last_undegraded":"2024-02-21T23:53:47.729386+0000","last_fullsized":"2024-02-21T23:53:47.729386+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'157","last_scrub_stamp":"2024-02-21T23:53:47.729316+0000","last_deep_scrub":"26'157","last_deep_scrub_stamp":"2024-02-21T23:53:47.729316+0000","last_clean_scrub_stamp":"2024-02-21T23:53:47.729316+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:00:49.740891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":11459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"26'29","reported_seq":74,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:46.280576+0000","last_change":"2024-02-21T23:53:46.280599+0000","last_active":"2024-02-21T23:53:46.280576+0000","last_peered":"2024-02-21T23:53:46.280576+0000","last_clean":"2024-02-21T23:53:46.280576+0000","last_became_active":"2024-02-21T23:25:17.138136+0000","last_became_peered":"2024-02-21T23:25:17.138136+0000","last_unstale":"2024-02-21T23:53:46.280576+0000","last_undegraded":"2024-02-21T23:53:46.280576+0000","last_fullsized":"2024-02-21T23:53:46.280576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'29","last_scrub_stamp":"2024-02-21T23:53:46.280506+0000","last_deep_scrub":"26'29","last_deep_scrub_stamp":"2024-02-21T23:53:46.280506+0000","last_clean_scrub_stamp":"2024-02-21T23:53:46.280506+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:07:05.299108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'8","reported_seq":55,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:42.156530+0000","last_change":"2024-02-21T23:53:42.156552+0000","last_active":"2024-02-21T23:53:42.156530+0000","last_peered":"2024-02-21T23:53:42.156530+0000","last_clean":"2024-02-21T23:53:42.156530+0000","last_became_active":"2024-02-21T23:25:16.129974+0000","last_became_peered":"2024-02-21T23:25:16.129974+0000","last_unstale":"2024-02-21T23:53:42.156530+0000","last_undegraded":"2024-02-21T23:53:42.156530+0000","last_fullsized":"2024-02-21T23:53:42.156530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'8","last_scrub_stamp":"2024-02-21T23:53:42.156458+0000","last_deep_scrub":"26'8","last_deep_scrub_stamp":"2024-02-21T23:53:42.156458+0000","last_clean_scrub_stamp":"2024-02-21T23:53:42.156458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:53:42.648367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"26'138","reported_seq":176,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:51.898941+0000","last_change":"2024-02-21T23:53:51.898986+0000","last_active":"2024-02-21T23:53:51.898941+0000","last_peered":"2024-02-21T23:53:51.898941+0000","last_clean":"2024-02-21T23:53:51.898941+0000","last_became_active":"2024-02-21T23:25:22.242960+0000","last_became_peered":"2024-02-21T23:25:22.242960+0000","last_unstale":"2024-02-21T23:53:51.898941+0000","last_undegraded":"2024-02-21T23:53:51.898941+0000","last_fullsized":"2024-02-21T23:53:51.898941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'138","last_scrub_stamp":"2024-02-21T23:53:51.898866+0000","last_deep_scrub":"26'138","last_deep_scrub_stamp":"2024-02-21T23:53:51.898866+0000","last_clean_scrub_stamp":"2024-02-21T23:53:51.898866+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:02.043385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":8319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"26'48","reported_seq":93,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:52.039091+0000","last_change":"2024-02-21T23:53:52.039134+0000","last_active":"2024-02-21T23:53:52.039091+0000","last_peered":"2024-02-21T23:53:52.039091+0000","last_clean":"2024-02-21T23:53:52.039091+0000","last_became_active":"2024-02-21T23:25:17.183980+0000","last_became_peered":"2024-02-21T23:25:17.183980+0000","last_unstale":"2024-02-21T23:53:52.039091+0000","last_undegraded":"2024-02-21T23:53:52.039091+0000","last_fullsized":"2024-02-21T23:53:52.039091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'48","last_scrub_stamp":"2024-02-21T23:53:52.039016+0000","last_deep_scrub":"26'48","last_deep_scrub_stamp":"2024-02-21T23:53:52.039016+0000","last_clean_scrub_stamp":"2024-02-21T23:53:52.039016+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:43:54.427434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"26'3","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:42.674132+0000","last_change":"2024-02-21T23:53:42.674154+0000","last_active":"2024-02-21T23:53:42.674132+0000","last_peered":"2024-02-21T23:53:42.674132+0000","last_clean":"2024-02-21T23:53:42.674132+0000","last_became_active":"2024-02-21T23:25:16.130553+0000","last_became_peered":"2024-02-21T23:25:16.130553+0000","last_unstale":"2024-02-21T23:53:42.674132+0000","last_undegraded":"2024-02-21T23:53:42.674132+0000","last_fullsized":"2024-02-21T23:53:42.674132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-02-21T23:53:42.674062+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-02-21T23:53:42.674062+0000","last_clean_scrub_stamp":"2024-02-21T23:53:42.674062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:59:47.865365+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'97","reported_seq":136,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:50.932894+0000","last_change":"2024-02-21T23:53:50.932936+0000","last_active":"2024-02-21T23:53:50.932894+0000","last_peered":"2024-02-21T23:53:50.932894+0000","last_clean":"2024-02-21T23:53:50.932894+0000","last_became_active":"2024-02-21T23:25:22.242900+0000","last_became_peered":"2024-02-21T23:25:22.242900+0000","last_unstale":"2024-02-21T23:53:50.932894+0000","last_undegraded":"2024-02-21T23:53:50.932894+0000","last_fullsized":"2024-02-21T23:53:50.932894+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'97","last_scrub_stamp":"2024-02-21T23:53:50.932795+0000","last_deep_scrub":"26'97","last_deep_scrub_stamp":"2024-02-21T23:53:50.932795+0000","last_clean_scrub_stamp":"2024-02-21T23:53:50.932795+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:00:52.894743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":7038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"26'44","reported_seq":89,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:42.666199+0000","last_change":"2024-02-21T23:53:42.666224+0000","last_active":"2024-02-21T23:53:42.666199+0000","last_peered":"2024-02-21T23:53:42.666199+0000","last_clean":"2024-02-21T23:53:42.666199+0000","last_became_active":"2024-02-21T23:25:17.183042+0000","last_became_peered":"2024-02-21T23:25:17.183042+0000","last_unstale":"2024-02-21T23:53:42.666199+0000","last_undegraded":"2024-02-21T23:53:42.666199+0000","last_fullsized":"2024-02-21T23:53:42.666199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'44","last_scrub_stamp":"2024-02-21T23:53:42.666121+0000","last_deep_scrub":"26'44","last_deep_scrub_stamp":"2024-02-21T23:53:42.666121+0000","last_clean_scrub_stamp":"2024-02-21T23:53:42.666121+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:34:16.384074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"26'253","reported_seq":300,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:37.623516+0000","last_change":"2024-02-21T23:53:37.623542+0000","last_active":"2024-02-21T23:53:37.623516+0000","last_peered":"2024-02-21T23:53:37.623516+0000","last_clean":"2024-02-21T23:53:37.623516+0000","last_became_active":"2024-02-21T23:25:16.127375+0000","last_became_peered":"2024-02-21T23:25:16.127375+0000","last_unstale":"2024-02-21T23:53:37.623516+0000","last_undegraded":"2024-02-21T23:53:37.623516+0000","last_fullsized":"2024-02-21T23:53:37.623516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'253","last_scrub_stamp":"2024-02-21T23:53:37.623451+0000","last_deep_scrub":"26'253","last_deep_scrub_stamp":"2024-02-21T23:53:37.623451+0000","last_clean_scrub_stamp":"2024-02-21T23:53:37.623451+0000","objects_scrubbed":3,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:33:06.807958+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2561517,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":2649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'148","reported_seq":186,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:54.631080+0000","last_change":"2024-02-21T23:53:54.631103+0000","last_active":"2024-02-21T23:53:54.631080+0000","last_peered":"2024-02-21T23:53:54.631080+0000","last_clean":"2024-02-21T23:53:54.631080+0000","last_became_active":"2024-02-21T23:25:22.247889+0000","last_became_peered":"2024-02-21T23:25:22.247889+0000","last_unstale":"2024-02-21T23:53:54.631080+0000","last_undegraded":"2024-02-21T23:53:54.631080+0000","last_fullsized":"2024-02-21T23:53:54.631080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'148","last_scrub_stamp":"2024-02-21T23:53:54.631009+0000","last_deep_scrub":"26'148","last_deep_scrub_stamp":"2024-02-21T23:53:54.631009+0000","last_clean_scrub_stamp":"2024-02-21T23:53:54.631009+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:04.775528+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":13204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"26'37","reported_seq":67,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:46.091392+0000","last_change":"2024-02-21T23:53:46.091421+0000","last_active":"2024-02-21T23:53:46.091392+0000","last_peered":"2024-02-21T23:53:46.091392+0000","last_clean":"2024-02-21T23:53:46.091392+0000","last_became_active":"2024-02-21T23:25:57.526874+0000","last_became_peered":"2024-02-21T23:25:57.526874+0000","last_unstale":"2024-02-21T23:53:46.091392+0000","last_undegraded":"2024-02-21T23:53:46.091392+0000","last_fullsized":"2024-02-21T23:53:46.091392+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'37","last_scrub_stamp":"2024-02-21T23:53:46.091324+0000","last_deep_scrub":"26'37","last_deep_scrub_stamp":"2024-02-21T23:53:46.091324+0000","last_clean_scrub_stamp":"2024-02-21T23:53:46.091324+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:54:00.955166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"26'838","reported_seq":885,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:40.096674+0000","last_change":"2024-02-21T23:53:40.096722+0000","last_active":"2024-02-21T23:53:40.096674+0000","last_peered":"2024-02-21T23:53:40.096674+0000","last_clean":"2024-02-21T23:53:40.096674+0000","last_became_active":"2024-02-21T23:25:16.116162+0000","last_became_peered":"2024-02-21T23:25:16.116162+0000","last_unstale":"2024-02-21T23:53:40.096674+0000","last_undegraded":"2024-02-21T23:53:40.096674+0000","last_fullsized":"2024-02-21T23:53:40.096674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'838","last_scrub_stamp":"2024-02-21T23:53:40.096574+0000","last_deep_scrub":"26'838","last_deep_scrub_stamp":"2024-02-21T23:53:40.096574+0000","last_clean_scrub_stamp":"2024-02-21T23:53:40.096574+0000","objects_scrubbed":4,"log_size":838,"log_dups_size":0,"ondisk_log_size":838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:42:42.152185+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"26'155","reported_seq":194,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:48.656837+0000","last_change":"2024-02-21T23:53:48.656859+0000","last_active":"2024-02-21T23:53:48.656837+0000","last_peered":"2024-02-21T23:53:48.656837+0000","last_clean":"2024-02-21T23:53:48.656837+0000","last_became_active":"2024-02-21T23:25:21.682425+0000","last_became_peered":"2024-02-21T23:25:21.682425+0000","last_unstale":"2024-02-21T23:53:48.656837+0000","last_undegraded":"2024-02-21T23:53:48.656837+0000","last_fullsized":"2024-02-21T23:53:48.656837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'155","last_scrub_stamp":"2024-02-21T23:53:48.656766+0000","last_deep_scrub":"26'155","last_deep_scrub_stamp":"2024-02-21T23:53:48.656766+0000","last_clean_scrub_stamp":"2024-02-21T23:53:48.656766+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:13.459700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":9421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"26'32","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:41.107204+0000","last_change":"2024-02-21T23:53:41.107225+0000","last_active":"2024-02-21T23:53:41.107204+0000","last_peered":"2024-02-21T23:53:41.107204+0000","last_clean":"2024-02-21T23:53:41.107204+0000","last_became_active":"2024-02-21T23:25:17.184574+0000","last_became_peered":"2024-02-21T23:25:17.184574+0000","last_unstale":"2024-02-21T23:53:41.107204+0000","last_undegraded":"2024-02-21T23:53:41.107204+0000","last_fullsized":"2024-02-21T23:53:41.107204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'32","last_scrub_stamp":"2024-02-21T23:53:41.107133+0000","last_deep_scrub":"26'32","last_deep_scrub_stamp":"2024-02-21T23:53:41.107133+0000","last_clean_scrub_stamp":"2024-02-21T23:53:41.107133+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:33:10.291640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"26'698","reported_seq":744,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:40.516923+0000","last_change":"2024-02-21T23:53:40.516964+0000","last_active":"2024-02-21T23:53:40.516923+0000","last_peered":"2024-02-21T23:53:40.516923+0000","last_clean":"2024-02-21T23:53:40.516923+0000","last_became_active":"2024-02-21T23:25:16.150767+0000","last_became_peered":"2024-02-21T23:25:16.150767+0000","last_unstale":"2024-02-21T23:53:40.516923+0000","last_undegraded":"2024-02-21T23:53:40.516923+0000","last_fullsized":"2024-02-21T23:53:40.516923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'698","last_scrub_stamp":"2024-02-21T23:53:40.516832+0000","last_deep_scrub":"26'698","last_deep_scrub_stamp":"2024-02-21T23:53:40.516832+0000","last_clean_scrub_stamp":"2024-02-21T23:53:40.516832+0000","objects_scrubbed":2,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:00:48.699403+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'152","reported_seq":191,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:47.117781+0000","last_change":"2024-02-21T23:53:47.117827+0000","last_active":"2024-02-21T23:53:47.117781+0000","last_peered":"2024-02-21T23:53:47.117781+0000","last_clean":"2024-02-21T23:53:47.117781+0000","last_became_active":"2024-02-21T23:25:21.683415+0000","last_became_peered":"2024-02-21T23:25:21.683415+0000","last_unstale":"2024-02-21T23:53:47.117781+0000","last_undegraded":"2024-02-21T23:53:47.117781+0000","last_fullsized":"2024-02-21T23:53:47.117781+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'152","last_scrub_stamp":"2024-02-21T23:53:47.117695+0000","last_deep_scrub":"26'152","last_deep_scrub_stamp":"2024-02-21T23:53:47.117695+0000","last_clean_scrub_stamp":"2024-02-21T23:53:47.117695+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:33:18.324522+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"26'42","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:44.646140+0000","last_change":"2024-02-21T23:53:44.646162+0000","last_active":"2024-02-21T23:53:44.646140+0000","last_peered":"2024-02-21T23:53:44.646140+0000","last_clean":"2024-02-21T23:53:44.646140+0000","last_became_active":"2024-02-21T23:25:17.186952+0000","last_became_peered":"2024-02-21T23:25:17.186952+0000","last_unstale":"2024-02-21T23:53:44.646140+0000","last_undegraded":"2024-02-21T23:53:44.646140+0000","last_fullsized":"2024-02-21T23:53:44.646140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'42","last_scrub_stamp":"2024-02-21T23:53:44.646073+0000","last_deep_scrub":"26'42","last_deep_scrub_stamp":"2024-02-21T23:53:44.646073+0000","last_clean_scrub_stamp":"2024-02-21T23:53:44.646073+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:57:09.224817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"26'2","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:44.646204+0000","last_change":"2024-02-21T23:53:44.646224+0000","last_active":"2024-02-21T23:53:44.646204+0000","last_peered":"2024-02-21T23:53:44.646204+0000","last_clean":"2024-02-21T23:53:44.646204+0000","last_became_active":"2024-02-21T23:25:16.134458+0000","last_became_peered":"2024-02-21T23:25:16.134458+0000","last_unstale":"2024-02-21T23:53:44.646204+0000","last_undegraded":"2024-02-21T23:53:44.646204+0000","last_fullsized":"2024-02-21T23:53:44.646204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-02-21T23:53:44.646140+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-02-21T23:53:44.646140+0000","last_clean_scrub_stamp":"2024-02-21T23:53:44.646140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:02:05.348643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"26'139","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:49.976200+0000","last_change":"2024-02-21T23:53:49.976222+0000","last_active":"2024-02-21T23:53:49.976200+0000","last_peered":"2024-02-21T23:53:49.976200+0000","last_clean":"2024-02-21T23:53:49.976200+0000","last_became_active":"2024-02-21T23:25:21.683705+0000","last_became_peered":"2024-02-21T23:25:21.683705+0000","last_unstale":"2024-02-21T23:53:49.976200+0000","last_undegraded":"2024-02-21T23:53:49.976200+0000","last_fullsized":"2024-02-21T23:53:49.976200+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'139","last_scrub_stamp":"2024-02-21T23:53:49.976134+0000","last_deep_scrub":"26'139","last_deep_scrub_stamp":"2024-02-21T23:53:49.976134+0000","last_clean_scrub_stamp":"2024-02-21T23:53:49.976134+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:48:18.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":10133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"26'42","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:39.693023+0000","last_change":"2024-02-21T23:53:39.693060+0000","last_active":"2024-02-21T23:53:39.693023+0000","last_peered":"2024-02-21T23:53:39.693023+0000","last_clean":"2024-02-21T23:53:39.693023+0000","last_became_active":"2024-02-21T23:25:17.187792+0000","last_became_peered":"2024-02-21T23:25:17.187792+0000","last_unstale":"2024-02-21T23:53:39.693023+0000","last_undegraded":"2024-02-21T23:53:39.693023+0000","last_fullsized":"2024-02-21T23:53:39.693023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'42","last_scrub_stamp":"2024-02-21T23:53:39.692933+0000","last_deep_scrub":"26'42","last_deep_scrub_stamp":"2024-02-21T23:53:39.692933+0000","last_clean_scrub_stamp":"2024-02-21T23:53:39.692933+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T11:33:08.877440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"26'5","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:41.118395+0000","last_change":"2024-02-21T23:53:41.118416+0000","last_active":"2024-02-21T23:53:41.118395+0000","last_peered":"2024-02-21T23:53:41.118395+0000","last_clean":"2024-02-21T23:53:41.118395+0000","last_became_active":"2024-02-21T23:25:16.129164+0000","last_became_peered":"2024-02-21T23:25:16.129164+0000","last_unstale":"2024-02-21T23:53:41.118395+0000","last_undegraded":"2024-02-21T23:53:41.118395+0000","last_fullsized":"2024-02-21T23:53:41.118395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-02-21T23:53:41.118329+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-02-21T23:53:41.118329+0000","last_clean_scrub_stamp":"2024-02-21T23:53:41.118329+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:33:48.600927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"26'139","reported_seq":172,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:47.042824+0000","last_change":"2024-02-21T23:53:47.042824+0000","last_active":"2024-02-21T23:53:47.042824+0000","last_peered":"2024-02-21T23:53:47.042824+0000","last_clean":"2024-02-21T23:53:47.042824+0000","last_became_active":"2024-02-21T23:25:57.599867+0000","last_became_peered":"2024-02-21T23:25:57.599867+0000","last_unstale":"2024-02-21T23:53:47.042824+0000","last_undegraded":"2024-02-21T23:53:47.042824+0000","last_fullsized":"2024-02-21T23:53:47.042824+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'139","last_scrub_stamp":"2024-02-21T23:53:47.042724+0000","last_deep_scrub":"26'139","last_deep_scrub_stamp":"2024-02-21T23:53:47.042724+0000","last_clean_scrub_stamp":"2024-02-21T23:53:47.042724+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:33:12.196335+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":5393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,5],"acting":[2,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"26'153","reported_seq":194,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:51.857453+0000","last_change":"2024-02-21T23:53:51.857477+0000","last_active":"2024-02-21T23:53:51.857453+0000","last_peered":"2024-02-21T23:53:51.857453+0000","last_clean":"2024-02-21T23:53:51.857453+0000","last_became_active":"2024-02-21T23:25:22.247811+0000","last_became_peered":"2024-02-21T23:25:22.247811+0000","last_unstale":"2024-02-21T23:53:51.857453+0000","last_undegraded":"2024-02-21T23:53:51.857453+0000","last_fullsized":"2024-02-21T23:53:51.857453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'153","last_scrub_stamp":"2024-02-21T23:53:51.857376+0000","last_deep_scrub":"26'153","last_deep_scrub_stamp":"2024-02-21T23:53:51.857376+0000","last_clean_scrub_stamp":"2024-02-21T23:53:51.857376+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:08:02.001895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":11393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"26'41","reported_seq":85,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:43.580303+0000","last_change":"2024-02-21T23:53:43.580324+0000","last_active":"2024-02-21T23:53:43.580303+0000","last_peered":"2024-02-21T23:53:43.580303+0000","last_clean":"2024-02-21T23:53:43.580303+0000","last_became_active":"2024-02-21T23:25:17.188885+0000","last_became_peered":"2024-02-21T23:25:17.188885+0000","last_unstale":"2024-02-21T23:53:43.580303+0000","last_undegraded":"2024-02-21T23:53:43.580303+0000","last_fullsized":"2024-02-21T23:53:43.580303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'41","last_scrub_stamp":"2024-02-21T23:53:43.580226+0000","last_deep_scrub":"26'41","last_deep_scrub_stamp":"2024-02-21T23:53:43.580226+0000","last_clean_scrub_stamp":"2024-02-21T23:53:43.580226+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:44:52.985117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"26'4","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:43.657367+0000","last_change":"2024-02-21T23:53:43.657405+0000","last_active":"2024-02-21T23:53:43.657367+0000","last_peered":"2024-02-21T23:53:43.657367+0000","last_clean":"2024-02-21T23:53:43.657367+0000","last_became_active":"2024-02-21T23:25:16.131756+0000","last_became_peered":"2024-02-21T23:25:16.131756+0000","last_unstale":"2024-02-21T23:53:43.657367+0000","last_undegraded":"2024-02-21T23:53:43.657367+0000","last_fullsized":"2024-02-21T23:53:43.657367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4","last_scrub_stamp":"2024-02-21T23:53:43.657283+0000","last_deep_scrub":"26'4","last_deep_scrub_stamp":"2024-02-21T23:53:43.657283+0000","last_clean_scrub_stamp":"2024-02-21T23:53:43.657283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:17:25.394469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"26'157","reported_seq":197,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:52.403402+0000","last_change":"2024-02-21T23:53:52.403450+0000","last_active":"2024-02-21T23:53:52.403402+0000","last_peered":"2024-02-21T23:53:52.403402+0000","last_clean":"2024-02-21T23:53:52.403402+0000","last_became_active":"2024-02-21T23:25:21.685677+0000","last_became_peered":"2024-02-21T23:25:21.685677+0000","last_unstale":"2024-02-21T23:53:52.403402+0000","last_undegraded":"2024-02-21T23:53:52.403402+0000","last_fullsized":"2024-02-21T23:53:52.403402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'157","last_scrub_stamp":"2024-02-21T23:53:52.403317+0000","last_deep_scrub":"26'157","last_deep_scrub_stamp":"2024-02-21T23:53:52.403317+0000","last_clean_scrub_stamp":"2024-02-21T23:53:52.403317+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:43:54.791735+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":15074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"26'44","reported_seq":88,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:44.934393+0000","last_change":"2024-02-21T23:53:44.934431+0000","last_active":"2024-02-21T23:53:44.934393+0000","last_peered":"2024-02-21T23:53:44.934393+0000","last_clean":"2024-02-21T23:53:44.934393+0000","last_became_active":"2024-02-21T23:25:17.134888+0000","last_became_peered":"2024-02-21T23:25:17.134888+0000","last_unstale":"2024-02-21T23:53:44.934393+0000","last_undegraded":"2024-02-21T23:53:44.934393+0000","last_fullsized":"2024-02-21T23:53:44.934393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'44","last_scrub_stamp":"2024-02-21T23:53:44.934275+0000","last_deep_scrub":"26'44","last_deep_scrub_stamp":"2024-02-21T23:53:44.934275+0000","last_clean_scrub_stamp":"2024-02-21T23:53:44.934275+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:30:34.028230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"26'5","reported_seq":51,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:40.920353+0000","last_change":"2024-02-21T23:53:40.920391+0000","last_active":"2024-02-21T23:53:40.920353+0000","last_peered":"2024-02-21T23:53:40.920353+0000","last_clean":"2024-02-21T23:53:40.920353+0000","last_became_active":"2024-02-21T23:25:16.116201+0000","last_became_peered":"2024-02-21T23:25:16.116201+0000","last_unstale":"2024-02-21T23:53:40.920353+0000","last_undegraded":"2024-02-21T23:53:40.920353+0000","last_fullsized":"2024-02-21T23:53:40.920353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5","last_scrub_stamp":"2024-02-21T23:53:40.920243+0000","last_deep_scrub":"26'5","last_deep_scrub_stamp":"2024-02-21T23:53:40.920243+0000","last_clean_scrub_stamp":"2024-02-21T23:53:40.920243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:17:22.657429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"26'719","reported_seq":765,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:43.181945+0000","last_change":"2024-02-21T23:53:43.181971+0000","last_active":"2024-02-21T23:53:43.181945+0000","last_peered":"2024-02-21T23:53:43.181945+0000","last_clean":"2024-02-21T23:53:43.181945+0000","last_became_active":"2024-02-21T23:25:16.132062+0000","last_became_peered":"2024-02-21T23:25:16.132062+0000","last_unstale":"2024-02-21T23:53:43.181945+0000","last_undegraded":"2024-02-21T23:53:43.181945+0000","last_fullsized":"2024-02-21T23:53:43.181945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'719","last_scrub_stamp":"2024-02-21T23:53:43.181876+0000","last_deep_scrub":"26'719","last_deep_scrub_stamp":"2024-02-21T23:53:43.181876+0000","last_clean_scrub_stamp":"2024-02-21T23:53:43.181876+0000","objects_scrubbed":1,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:33:08.335487+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":716,"num_write_kb":4453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"26'39","reported_seq":83,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:40.632496+0000","last_change":"2024-02-21T23:53:40.632517+0000","last_active":"2024-02-21T23:53:40.632496+0000","last_peered":"2024-02-21T23:53:40.632496+0000","last_clean":"2024-02-21T23:53:40.632496+0000","last_became_active":"2024-02-21T23:25:17.161176+0000","last_became_peered":"2024-02-21T23:25:17.161176+0000","last_unstale":"2024-02-21T23:53:40.632496+0000","last_undegraded":"2024-02-21T23:53:40.632496+0000","last_fullsized":"2024-02-21T23:53:40.632496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'39","last_scrub_stamp":"2024-02-21T23:53:40.632434+0000","last_deep_scrub":"26'39","last_deep_scrub_stamp":"2024-02-21T23:53:40.632434+0000","last_clean_scrub_stamp":"2024-02-21T23:53:40.632434+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:06:59.651035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"26'95","reported_seq":133,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:43.083812+0000","last_change":"2024-02-21T23:53:43.083858+0000","last_active":"2024-02-21T23:53:43.083812+0000","last_peered":"2024-02-21T23:53:43.083812+0000","last_clean":"2024-02-21T23:53:43.083812+0000","last_became_active":"2024-02-21T23:25:21.685375+0000","last_became_peered":"2024-02-21T23:25:21.685375+0000","last_unstale":"2024-02-21T23:53:43.083812+0000","last_undegraded":"2024-02-21T23:53:43.083812+0000","last_fullsized":"2024-02-21T23:53:43.083812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'95","last_scrub_stamp":"2024-02-21T23:53:43.083711+0000","last_deep_scrub":"26'95","last_deep_scrub_stamp":"2024-02-21T23:53:43.083711+0000","last_clean_scrub_stamp":"2024-02-21T23:53:43.083711+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:46:18.914080+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":7997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"26'689","reported_seq":735,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:42.989944+0000","last_change":"2024-02-21T23:53:42.989971+0000","last_active":"2024-02-21T23:53:42.989944+0000","last_peered":"2024-02-21T23:53:42.989944+0000","last_clean":"2024-02-21T23:53:42.989944+0000","last_became_active":"2024-02-21T23:25:16.133839+0000","last_became_peered":"2024-02-21T23:25:16.133839+0000","last_unstale":"2024-02-21T23:53:42.989944+0000","last_undegraded":"2024-02-21T23:53:42.989944+0000","last_fullsized":"2024-02-21T23:53:42.989944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'689","last_scrub_stamp":"2024-02-21T23:53:42.989870+0000","last_deep_scrub":"26'689","last_deep_scrub_stamp":"2024-02-21T23:53:42.989870+0000","last_clean_scrub_stamp":"2024-02-21T23:53:42.989870+0000","objects_scrubbed":1,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:44:49.916673+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"26'33","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:43.096539+0000","last_change":"2024-02-21T23:53:43.096578+0000","last_active":"2024-02-21T23:53:43.096539+0000","last_peered":"2024-02-21T23:53:43.096539+0000","last_clean":"2024-02-21T23:53:43.096539+0000","last_became_active":"2024-02-21T23:25:17.190194+0000","last_became_peered":"2024-02-21T23:25:17.190194+0000","last_unstale":"2024-02-21T23:53:43.096539+0000","last_undegraded":"2024-02-21T23:53:43.096539+0000","last_fullsized":"2024-02-21T23:53:43.096539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'33","last_scrub_stamp":"2024-02-21T23:53:43.096450+0000","last_deep_scrub":"26'33","last_deep_scrub_stamp":"2024-02-21T23:53:43.096450+0000","last_clean_scrub_stamp":"2024-02-21T23:53:43.096450+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:34:04.318831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"26'159","reported_seq":204,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:54.849354+0000","last_change":"2024-02-21T23:53:54.849396+0000","last_active":"2024-02-21T23:53:54.849354+0000","last_peered":"2024-02-21T23:53:54.849354+0000","last_clean":"2024-02-21T23:53:54.849354+0000","last_became_active":"2024-02-21T23:25:57.525755+0000","last_became_peered":"2024-02-21T23:25:57.525755+0000","last_unstale":"2024-02-21T23:53:54.849354+0000","last_undegraded":"2024-02-21T23:53:54.849354+0000","last_fullsized":"2024-02-21T23:53:54.849354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'159","last_scrub_stamp":"2024-02-21T23:53:54.849246+0000","last_deep_scrub":"26'159","last_deep_scrub_stamp":"2024-02-21T23:53:54.849246+0000","last_clean_scrub_stamp":"2024-02-21T23:53:54.849246+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:46:26.872922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4,2],"acting":[3,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"26'48","reported_seq":94,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:45.251914+0000","last_change":"2024-02-21T23:53:45.251945+0000","last_active":"2024-02-21T23:53:45.251914+0000","last_peered":"2024-02-21T23:53:45.251914+0000","last_clean":"2024-02-21T23:53:45.251914+0000","last_became_active":"2024-02-21T23:25:17.137717+0000","last_became_peered":"2024-02-21T23:25:17.137717+0000","last_unstale":"2024-02-21T23:53:45.251914+0000","last_undegraded":"2024-02-21T23:53:45.251914+0000","last_fullsized":"2024-02-21T23:53:45.251914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'48","last_scrub_stamp":"2024-02-21T23:53:45.251844+0000","last_deep_scrub":"26'48","last_deep_scrub_stamp":"2024-02-21T23:53:45.251844+0000","last_clean_scrub_stamp":"2024-02-21T23:53:45.251844+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:34:06.474225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"26'48","reported_seq":94,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:42.093620+0000","last_change":"2024-02-21T23:53:42.093676+0000","last_active":"2024-02-21T23:53:42.093620+0000","last_peered":"2024-02-21T23:53:42.093620+0000","last_clean":"2024-02-21T23:53:42.093620+0000","last_became_active":"2024-02-21T23:25:17.194533+0000","last_became_peered":"2024-02-21T23:25:17.194533+0000","last_unstale":"2024-02-21T23:53:42.093620+0000","last_undegraded":"2024-02-21T23:53:42.093620+0000","last_fullsized":"2024-02-21T23:53:42.093620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'48","last_scrub_stamp":"2024-02-21T23:53:42.093487+0000","last_deep_scrub":"26'48","last_deep_scrub_stamp":"2024-02-21T23:53:42.093487+0000","last_clean_scrub_stamp":"2024-02-21T23:53:42.093487+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T07:26:04.995641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"26'50","reported_seq":96,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:50.545800+0000","last_change":"2024-02-21T23:53:50.545837+0000","last_active":"2024-02-21T23:53:50.545800+0000","last_peered":"2024-02-21T23:53:50.545800+0000","last_clean":"2024-02-21T23:53:50.545800+0000","last_became_active":"2024-02-21T23:25:17.190600+0000","last_became_peered":"2024-02-21T23:25:17.190600+0000","last_unstale":"2024-02-21T23:53:50.545800+0000","last_undegraded":"2024-02-21T23:53:50.545800+0000","last_fullsized":"2024-02-21T23:53:50.545800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'50","last_scrub_stamp":"2024-02-21T23:53:50.545715+0000","last_deep_scrub":"26'50","last_deep_scrub_stamp":"2024-02-21T23:53:50.545715+0000","last_clean_scrub_stamp":"2024-02-21T23:53:50.545715+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:21:22.831267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"26'36","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:39.608179+0000","last_change":"2024-02-21T23:53:39.608217+0000","last_active":"2024-02-21T23:53:39.608179+0000","last_peered":"2024-02-21T23:53:39.608179+0000","last_clean":"2024-02-21T23:53:39.608179+0000","last_became_active":"2024-02-21T23:25:17.135011+0000","last_became_peered":"2024-02-21T23:25:17.135011+0000","last_unstale":"2024-02-21T23:53:39.608179+0000","last_undegraded":"2024-02-21T23:53:39.608179+0000","last_fullsized":"2024-02-21T23:53:39.608179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'36","last_scrub_stamp":"2024-02-21T23:53:39.608081+0000","last_deep_scrub":"26'36","last_deep_scrub_stamp":"2024-02-21T23:53:39.608081+0000","last_clean_scrub_stamp":"2024-02-21T23:53:39.608081+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:34:00.830462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"26'29","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:38.014008+0000","last_change":"2024-02-21T23:53:38.014030+0000","last_active":"2024-02-21T23:53:38.014008+0000","last_peered":"2024-02-21T23:53:38.014008+0000","last_clean":"2024-02-21T23:53:38.014008+0000","last_became_active":"2024-02-21T23:25:17.161401+0000","last_became_peered":"2024-02-21T23:25:17.161401+0000","last_unstale":"2024-02-21T23:53:38.014008+0000","last_undegraded":"2024-02-21T23:53:38.014008+0000","last_fullsized":"2024-02-21T23:53:38.014008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'29","last_scrub_stamp":"2024-02-21T23:53:38.013931+0000","last_deep_scrub":"26'29","last_deep_scrub_stamp":"2024-02-21T23:53:38.013931+0000","last_clean_scrub_stamp":"2024-02-21T23:53:38.013931+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:42.703272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"26'34","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:45.595645+0000","last_change":"2024-02-21T23:53:45.595668+0000","last_active":"2024-02-21T23:53:45.595645+0000","last_peered":"2024-02-21T23:53:45.595645+0000","last_clean":"2024-02-21T23:53:45.595645+0000","last_became_active":"2024-02-21T23:25:17.189484+0000","last_became_peered":"2024-02-21T23:25:17.189484+0000","last_unstale":"2024-02-21T23:53:45.595645+0000","last_undegraded":"2024-02-21T23:53:45.595645+0000","last_fullsized":"2024-02-21T23:53:45.595645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'34","last_scrub_stamp":"2024-02-21T23:53:45.595577+0000","last_deep_scrub":"26'34","last_deep_scrub_stamp":"2024-02-21T23:53:45.595577+0000","last_clean_scrub_stamp":"2024-02-21T23:53:45.595577+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:58:11.787377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"26'45","reported_seq":91,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:49.334498+0000","last_change":"2024-02-21T23:53:49.334524+0000","last_active":"2024-02-21T23:53:49.334498+0000","last_peered":"2024-02-21T23:53:49.334498+0000","last_clean":"2024-02-21T23:53:49.334498+0000","last_became_active":"2024-02-21T23:25:17.183865+0000","last_became_peered":"2024-02-21T23:25:17.183865+0000","last_unstale":"2024-02-21T23:53:49.334498+0000","last_undegraded":"2024-02-21T23:53:49.334498+0000","last_fullsized":"2024-02-21T23:53:49.334498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'45","last_scrub_stamp":"2024-02-21T23:53:49.334424+0000","last_deep_scrub":"26'45","last_deep_scrub_stamp":"2024-02-21T23:53:49.334424+0000","last_clean_scrub_stamp":"2024-02-21T23:53:49.334424+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:33:20.541251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"26'37","reported_seq":83,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:39.944127+0000","last_change":"2024-02-21T23:53:39.944181+0000","last_active":"2024-02-21T23:53:39.944127+0000","last_peered":"2024-02-21T23:53:39.944127+0000","last_clean":"2024-02-21T23:53:39.944127+0000","last_became_active":"2024-02-21T23:25:17.161198+0000","last_became_peered":"2024-02-21T23:25:17.161198+0000","last_unstale":"2024-02-21T23:53:39.944127+0000","last_undegraded":"2024-02-21T23:53:39.944127+0000","last_fullsized":"2024-02-21T23:53:39.944127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'37","last_scrub_stamp":"2024-02-21T23:53:39.944045+0000","last_deep_scrub":"26'37","last_deep_scrub_stamp":"2024-02-21T23:53:39.944045+0000","last_clean_scrub_stamp":"2024-02-21T23:53:39.944045+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:06:58.962646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"26'32","reported_seq":78,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:48.541849+0000","last_change":"2024-02-21T23:53:48.541892+0000","last_active":"2024-02-21T23:53:48.541849+0000","last_peered":"2024-02-21T23:53:48.541849+0000","last_clean":"2024-02-21T23:53:48.541849+0000","last_became_active":"2024-02-21T23:25:17.185420+0000","last_became_peered":"2024-02-21T23:25:17.185420+0000","last_unstale":"2024-02-21T23:53:48.541849+0000","last_undegraded":"2024-02-21T23:53:48.541849+0000","last_fullsized":"2024-02-21T23:53:48.541849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'32","last_scrub_stamp":"2024-02-21T23:53:48.541740+0000","last_deep_scrub":"26'32","last_deep_scrub_stamp":"2024-02-21T23:53:48.541740+0000","last_clean_scrub_stamp":"2024-02-21T23:53:48.541740+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:43:50.930159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"26'46","reported_seq":91,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:48.000216+0000","last_change":"2024-02-21T23:53:48.000238+0000","last_active":"2024-02-21T23:53:48.000216+0000","last_peered":"2024-02-21T23:53:48.000216+0000","last_clean":"2024-02-21T23:53:48.000216+0000","last_became_active":"2024-02-21T23:25:17.161010+0000","last_became_peered":"2024-02-21T23:25:17.161010+0000","last_unstale":"2024-02-21T23:53:48.000216+0000","last_undegraded":"2024-02-21T23:53:48.000216+0000","last_fullsized":"2024-02-21T23:53:48.000216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'46","last_scrub_stamp":"2024-02-21T23:53:48.000146+0000","last_deep_scrub":"26'46","last_deep_scrub_stamp":"2024-02-21T23:53:48.000146+0000","last_clean_scrub_stamp":"2024-02-21T23:53:48.000146+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:21:20.285698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"26'39","reported_seq":84,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:51.572622+0000","last_change":"2024-02-21T23:53:51.572643+0000","last_active":"2024-02-21T23:53:51.572622+0000","last_peered":"2024-02-21T23:53:51.572622+0000","last_clean":"2024-02-21T23:53:51.572622+0000","last_became_active":"2024-02-21T23:25:17.187741+0000","last_became_peered":"2024-02-21T23:25:17.187741+0000","last_unstale":"2024-02-21T23:53:51.572622+0000","last_undegraded":"2024-02-21T23:53:51.572622+0000","last_fullsized":"2024-02-21T23:53:51.572622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'39","last_scrub_stamp":"2024-02-21T23:53:51.572554+0000","last_deep_scrub":"26'39","last_deep_scrub_stamp":"2024-02-21T23:53:51.572554+0000","last_clean_scrub_stamp":"2024-02-21T23:53:51.572554+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:21:24.350014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"26'53","reported_seq":98,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:43.640110+0000","last_change":"2024-02-21T23:53:43.640131+0000","last_active":"2024-02-21T23:53:43.640110+0000","last_peered":"2024-02-21T23:53:43.640110+0000","last_clean":"2024-02-21T23:53:43.640110+0000","last_became_active":"2024-02-21T23:25:17.186108+0000","last_became_peered":"2024-02-21T23:25:17.186108+0000","last_unstale":"2024-02-21T23:53:43.640110+0000","last_undegraded":"2024-02-21T23:53:43.640110+0000","last_fullsized":"2024-02-21T23:53:43.640110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'53","last_scrub_stamp":"2024-02-21T23:53:43.640032+0000","last_deep_scrub":"26'53","last_deep_scrub_stamp":"2024-02-21T23:53:43.640032+0000","last_clean_scrub_stamp":"2024-02-21T23:53:43.640032+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:33:14.846859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"26'43","reported_seq":88,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:47.313167+0000","last_change":"2024-02-21T23:53:47.313189+0000","last_active":"2024-02-21T23:53:47.313167+0000","last_peered":"2024-02-21T23:53:47.313167+0000","last_clean":"2024-02-21T23:53:47.313167+0000","last_became_active":"2024-02-21T23:25:17.161140+0000","last_became_peered":"2024-02-21T23:25:17.161140+0000","last_unstale":"2024-02-21T23:53:47.313167+0000","last_undegraded":"2024-02-21T23:53:47.313167+0000","last_fullsized":"2024-02-21T23:53:47.313167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'43","last_scrub_stamp":"2024-02-21T23:53:47.313084+0000","last_deep_scrub":"26'43","last_deep_scrub_stamp":"2024-02-21T23:53:47.313084+0000","last_clean_scrub_stamp":"2024-02-21T23:53:47.313084+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:50:07.904460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"26'36","reported_seq":81,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:55.908514+0000","last_change":"2024-02-21T23:53:55.908553+0000","last_active":"2024-02-21T23:53:55.908514+0000","last_peered":"2024-02-21T23:53:55.908514+0000","last_clean":"2024-02-21T23:53:55.908514+0000","last_became_active":"2024-02-21T23:25:17.190420+0000","last_became_peered":"2024-02-21T23:25:17.190420+0000","last_unstale":"2024-02-21T23:53:55.908514+0000","last_undegraded":"2024-02-21T23:53:55.908514+0000","last_fullsized":"2024-02-21T23:53:55.908514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'36","last_scrub_stamp":"2024-02-21T23:53:55.908410+0000","last_deep_scrub":"26'36","last_deep_scrub_stamp":"2024-02-21T23:53:55.908410+0000","last_clean_scrub_stamp":"2024-02-21T23:53:55.908410+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:40:37.937398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"26'48","reported_seq":93,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:41.138324+0000","last_change":"2024-02-21T23:53:41.138345+0000","last_active":"2024-02-21T23:53:41.138324+0000","last_peered":"2024-02-21T23:53:41.138324+0000","last_clean":"2024-02-21T23:53:41.138324+0000","last_became_active":"2024-02-21T23:25:17.189302+0000","last_became_peered":"2024-02-21T23:25:17.189302+0000","last_unstale":"2024-02-21T23:53:41.138324+0000","last_undegraded":"2024-02-21T23:53:41.138324+0000","last_fullsized":"2024-02-21T23:53:41.138324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'48","last_scrub_stamp":"2024-02-21T23:53:41.138246+0000","last_deep_scrub":"26'48","last_deep_scrub_stamp":"2024-02-21T23:53:41.138246+0000","last_clean_scrub_stamp":"2024-02-21T23:53:41.138246+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:57:30.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"26'41","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:40.722580+0000","last_change":"2024-02-21T23:53:40.722601+0000","last_active":"2024-02-21T23:53:40.722580+0000","last_peered":"2024-02-21T23:53:40.722580+0000","last_clean":"2024-02-21T23:53:40.722580+0000","last_became_active":"2024-02-21T23:25:17.187811+0000","last_became_peered":"2024-02-21T23:25:17.187811+0000","last_unstale":"2024-02-21T23:53:40.722580+0000","last_undegraded":"2024-02-21T23:53:40.722580+0000","last_fullsized":"2024-02-21T23:53:40.722580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'41","last_scrub_stamp":"2024-02-21T23:53:40.722515+0000","last_deep_scrub":"26'41","last_deep_scrub_stamp":"2024-02-21T23:53:40.722515+0000","last_clean_scrub_stamp":"2024-02-21T23:53:40.722515+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T02:41:45.411855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"26'44","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:44.073999+0000","last_change":"2024-02-21T23:53:44.073999+0000","last_active":"2024-02-21T23:53:44.073999+0000","last_peered":"2024-02-21T23:53:44.073999+0000","last_clean":"2024-02-21T23:53:44.073999+0000","last_became_active":"2024-02-21T23:25:57.525443+0000","last_became_peered":"2024-02-21T23:25:57.525443+0000","last_unstale":"2024-02-21T23:53:44.073999+0000","last_undegraded":"2024-02-21T23:53:44.073999+0000","last_fullsized":"2024-02-21T23:53:44.073999+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'44","last_scrub_stamp":"2024-02-21T23:53:44.073897+0000","last_deep_scrub":"26'44","last_deep_scrub_stamp":"2024-02-21T23:53:44.073897+0000","last_clean_scrub_stamp":"2024-02-21T23:53:44.073897+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:35:19.123697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"26'38","reported_seq":83,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:42.556076+0000","last_change":"2024-02-21T23:53:42.556114+0000","last_active":"2024-02-21T23:53:42.556076+0000","last_peered":"2024-02-21T23:53:42.556076+0000","last_clean":"2024-02-21T23:53:42.556076+0000","last_became_active":"2024-02-21T23:25:17.186086+0000","last_became_peered":"2024-02-21T23:25:17.186086+0000","last_unstale":"2024-02-21T23:53:42.556076+0000","last_undegraded":"2024-02-21T23:53:42.556076+0000","last_fullsized":"2024-02-21T23:53:42.556076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'38","last_scrub_stamp":"2024-02-21T23:53:42.555970+0000","last_deep_scrub":"26'38","last_deep_scrub_stamp":"2024-02-21T23:53:42.555970+0000","last_clean_scrub_stamp":"2024-02-21T23:53:42.555970+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:50:03.147346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"26'41","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:36.598926+0000","last_change":"2024-02-21T23:53:36.598948+0000","last_active":"2024-02-21T23:53:36.598926+0000","last_peered":"2024-02-21T23:53:36.598926+0000","last_clean":"2024-02-21T23:53:36.598926+0000","last_became_active":"2024-02-21T23:25:17.190680+0000","last_became_peered":"2024-02-21T23:25:17.190680+0000","last_unstale":"2024-02-21T23:53:36.598926+0000","last_undegraded":"2024-02-21T23:53:36.598926+0000","last_fullsized":"2024-02-21T23:53:36.598926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'41","last_scrub_stamp":"2024-02-21T23:53:36.598855+0000","last_deep_scrub":"26'41","last_deep_scrub_stamp":"2024-02-21T23:53:36.598855+0000","last_clean_scrub_stamp":"2024-02-21T23:53:36.598855+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:03:03.021728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"26'38","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:43.934710+0000","last_change":"2024-02-21T23:53:43.934734+0000","last_active":"2024-02-21T23:53:43.934710+0000","last_peered":"2024-02-21T23:53:43.934710+0000","last_clean":"2024-02-21T23:53:43.934710+0000","last_became_active":"2024-02-21T23:25:17.134866+0000","last_became_peered":"2024-02-21T23:25:17.134866+0000","last_unstale":"2024-02-21T23:53:43.934710+0000","last_undegraded":"2024-02-21T23:53:43.934710+0000","last_fullsized":"2024-02-21T23:53:43.934710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'38","last_scrub_stamp":"2024-02-21T23:53:43.934602+0000","last_deep_scrub":"26'38","last_deep_scrub_stamp":"2024-02-21T23:53:43.934602+0000","last_clean_scrub_stamp":"2024-02-21T23:53:43.934602+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T06:37:04.191763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"26'27","reported_seq":71,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:39.061754+0000","last_change":"2024-02-21T23:53:39.061791+0000","last_active":"2024-02-21T23:53:39.061754+0000","last_peered":"2024-02-21T23:53:39.061754+0000","last_clean":"2024-02-21T23:53:39.061754+0000","last_became_active":"2024-02-21T23:25:17.161807+0000","last_became_peered":"2024-02-21T23:25:17.161807+0000","last_unstale":"2024-02-21T23:53:39.061754+0000","last_undegraded":"2024-02-21T23:53:39.061754+0000","last_fullsized":"2024-02-21T23:53:39.061754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'27","last_scrub_stamp":"2024-02-21T23:53:39.061677+0000","last_deep_scrub":"26'27","last_deep_scrub_stamp":"2024-02-21T23:53:39.061677+0000","last_clean_scrub_stamp":"2024-02-21T23:53:39.061677+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T08:07:04.074598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"26'36","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:51.364977+0000","last_change":"2024-02-21T23:53:51.365019+0000","last_active":"2024-02-21T23:53:51.364977+0000","last_peered":"2024-02-21T23:53:51.364977+0000","last_clean":"2024-02-21T23:53:51.364977+0000","last_became_active":"2024-02-21T23:25:17.188646+0000","last_became_peered":"2024-02-21T23:25:17.188646+0000","last_unstale":"2024-02-21T23:53:51.364977+0000","last_undegraded":"2024-02-21T23:53:51.364977+0000","last_fullsized":"2024-02-21T23:53:51.364977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'36","last_scrub_stamp":"2024-02-21T23:53:51.364891+0000","last_deep_scrub":"26'36","last_deep_scrub_stamp":"2024-02-21T23:53:51.364891+0000","last_clean_scrub_stamp":"2024-02-21T23:53:51.364891+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:30:40.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"26'51","reported_seq":95,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:50.349819+0000","last_change":"2024-02-21T23:53:50.349840+0000","last_active":"2024-02-21T23:53:50.349819+0000","last_peered":"2024-02-21T23:53:50.349819+0000","last_clean":"2024-02-21T23:53:50.349819+0000","last_became_active":"2024-02-21T23:25:17.188066+0000","last_became_peered":"2024-02-21T23:25:17.188066+0000","last_unstale":"2024-02-21T23:53:50.349819+0000","last_undegraded":"2024-02-21T23:53:50.349819+0000","last_fullsized":"2024-02-21T23:53:50.349819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'51","last_scrub_stamp":"2024-02-21T23:53:50.349739+0000","last_deep_scrub":"26'51","last_deep_scrub_stamp":"2024-02-21T23:53:50.349739+0000","last_clean_scrub_stamp":"2024-02-21T23:53:50.349739+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T09:58:16.541540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"26'39","reported_seq":83,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:53.944141+0000","last_change":"2024-02-21T23:53:53.944179+0000","last_active":"2024-02-21T23:53:53.944141+0000","last_peered":"2024-02-21T23:53:53.944141+0000","last_clean":"2024-02-21T23:53:53.944141+0000","last_became_active":"2024-02-21T23:25:17.187297+0000","last_became_peered":"2024-02-21T23:25:17.187297+0000","last_unstale":"2024-02-21T23:53:53.944141+0000","last_undegraded":"2024-02-21T23:53:53.944141+0000","last_fullsized":"2024-02-21T23:53:53.944141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'39","last_scrub_stamp":"2024-02-21T23:53:53.944056+0000","last_deep_scrub":"26'39","last_deep_scrub_stamp":"2024-02-21T23:53:53.944056+0000","last_clean_scrub_stamp":"2024-02-21T23:53:53.944056+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:21:26.229607+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"26'30","reported_seq":60,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:45.071826+0000","last_change":"2024-02-21T23:53:45.071848+0000","last_active":"2024-02-21T23:53:45.071826+0000","last_peered":"2024-02-21T23:53:45.071826+0000","last_clean":"2024-02-21T23:53:45.071826+0000","last_became_active":"2024-02-21T23:25:57.526068+0000","last_became_peered":"2024-02-21T23:25:57.526068+0000","last_unstale":"2024-02-21T23:53:45.071826+0000","last_undegraded":"2024-02-21T23:53:45.071826+0000","last_fullsized":"2024-02-21T23:53:45.071826+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'30","last_scrub_stamp":"2024-02-21T23:53:45.071757+0000","last_deep_scrub":"26'30","last_deep_scrub_stamp":"2024-02-21T23:53:45.071757+0000","last_clean_scrub_stamp":"2024-02-21T23:53:45.071757+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T00:33:52.554354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"26'37","reported_seq":81,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:49.112201+0000","last_change":"2024-02-21T23:53:49.112240+0000","last_active":"2024-02-21T23:53:49.112201+0000","last_peered":"2024-02-21T23:53:49.112201+0000","last_clean":"2024-02-21T23:53:49.112201+0000","last_became_active":"2024-02-21T23:25:17.182924+0000","last_became_peered":"2024-02-21T23:25:17.182924+0000","last_unstale":"2024-02-21T23:53:49.112201+0000","last_undegraded":"2024-02-21T23:53:49.112201+0000","last_fullsized":"2024-02-21T23:53:49.112201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'37","last_scrub_stamp":"2024-02-21T23:53:49.112121+0000","last_deep_scrub":"26'37","last_deep_scrub_stamp":"2024-02-21T23:53:49.112121+0000","last_clean_scrub_stamp":"2024-02-21T23:53:49.112121+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T03:44:56.038924+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"26'38","reported_seq":82,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:49.590902+0000","last_change":"2024-02-21T23:53:49.590923+0000","last_active":"2024-02-21T23:53:49.590902+0000","last_peered":"2024-02-21T23:53:49.590902+0000","last_clean":"2024-02-21T23:53:49.590902+0000","last_became_active":"2024-02-21T23:25:17.190276+0000","last_became_peered":"2024-02-21T23:25:17.190276+0000","last_unstale":"2024-02-21T23:53:49.590902+0000","last_undegraded":"2024-02-21T23:53:49.590902+0000","last_fullsized":"2024-02-21T23:53:49.590902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'38","last_scrub_stamp":"2024-02-21T23:53:49.590842+0000","last_deep_scrub":"26'38","last_deep_scrub_stamp":"2024-02-21T23:53:49.590842+0000","last_clean_scrub_stamp":"2024-02-21T23:53:49.590842+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T01:25:27.029040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"26'36","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:54.917324+0000","last_change":"2024-02-21T23:53:54.917345+0000","last_active":"2024-02-21T23:53:54.917324+0000","last_peered":"2024-02-21T23:53:54.917324+0000","last_clean":"2024-02-21T23:53:54.917324+0000","last_became_active":"2024-02-21T23:25:17.189614+0000","last_became_peered":"2024-02-21T23:25:17.189614+0000","last_unstale":"2024-02-21T23:53:54.917324+0000","last_undegraded":"2024-02-21T23:53:54.917324+0000","last_fullsized":"2024-02-21T23:53:54.917324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'36","last_scrub_stamp":"2024-02-21T23:53:54.917255+0000","last_deep_scrub":"26'36","last_deep_scrub_stamp":"2024-02-21T23:53:54.917255+0000","last_clean_scrub_stamp":"2024-02-21T23:53:54.917255+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:21:27.694716+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"26'25","reported_seq":77,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:56.566445+0000","last_change":"2024-02-21T23:53:56.566482+0000","last_active":"2024-02-21T23:53:56.566445+0000","last_peered":"2024-02-21T23:53:56.566445+0000","last_clean":"2024-02-21T23:53:56.566445+0000","last_became_active":"2024-02-21T23:25:57.600094+0000","last_became_peered":"2024-02-21T23:25:57.600094+0000","last_unstale":"2024-02-21T23:53:56.566445+0000","last_undegraded":"2024-02-21T23:53:56.566445+0000","last_fullsized":"2024-02-21T23:53:56.566445+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'25","last_scrub_stamp":"2024-02-21T23:53:56.566373+0000","last_deep_scrub":"26'25","last_deep_scrub_stamp":"2024-02-21T23:53:56.566373+0000","last_clean_scrub_stamp":"2024-02-21T23:53:56.566373+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:55:19.185138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"26'44","reported_seq":90,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:35.558393+0000","last_change":"2024-02-21T23:53:35.558434+0000","last_active":"2024-02-21T23:53:35.558393+0000","last_peered":"2024-02-21T23:53:35.558393+0000","last_clean":"2024-02-21T23:53:35.558393+0000","last_became_active":"2024-02-21T23:25:17.190576+0000","last_became_peered":"2024-02-21T23:25:17.190576+0000","last_unstale":"2024-02-21T23:53:35.558393+0000","last_undegraded":"2024-02-21T23:53:35.558393+0000","last_fullsized":"2024-02-21T23:53:35.558393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'44","last_scrub_stamp":"2024-02-21T23:53:35.558280+0000","last_deep_scrub":"26'44","last_deep_scrub_stamp":"2024-02-21T23:53:35.558280+0000","last_clean_scrub_stamp":"2024-02-21T23:53:35.558280+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T10:53:50.422122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"26'34","reported_seq":80,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:44.554490+0000","last_change":"2024-02-21T23:53:44.554513+0000","last_active":"2024-02-21T23:53:44.554490+0000","last_peered":"2024-02-21T23:53:44.554490+0000","last_clean":"2024-02-21T23:53:44.554490+0000","last_became_active":"2024-02-21T23:25:17.189029+0000","last_became_peered":"2024-02-21T23:25:17.189029+0000","last_unstale":"2024-02-21T23:53:44.554490+0000","last_undegraded":"2024-02-21T23:53:44.554490+0000","last_fullsized":"2024-02-21T23:53:44.554490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'34","last_scrub_stamp":"2024-02-21T23:53:44.554414+0000","last_deep_scrub":"26'34","last_deep_scrub_stamp":"2024-02-21T23:53:44.554414+0000","last_clean_scrub_stamp":"2024-02-21T23:53:44.554414+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T04:33:15.761241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"26'29","reported_seq":75,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-02-21T23:53:51.057676+0000","last_change":"2024-02-21T23:53:51.057717+0000","last_active":"2024-02-21T23:53:51.057676+0000","last_peered":"2024-02-21T23:53:51.057676+0000","last_clean":"2024-02-21T23:53:51.057676+0000","last_became_active":"2024-02-21T23:25:17.183410+0000","last_became_peered":"2024-02-21T23:25:17.183410+0000","last_unstale":"2024-02-21T23:53:51.057676+0000","last_undegraded":"2024-02-21T23:53:51.057676+0000","last_fullsized":"2024-02-21T23:53:51.057676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'29","last_scrub_stamp":"2024-02-21T23:53:51.057595+0000","last_deep_scrub":"26'29","last_deep_scrub_stamp":"2024-02-21T23:53:51.057595+0000","last_clean_scrub_stamp":"2024-02-21T23:53:51.057595+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-23T05:30:40.151550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4087,"num_write_kb":323834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4616,"ondisk_log_size":4616,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2419,"ondisk_log_size":2419,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":40532172,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":994,"num_read_kb":1060,"num_write":11557,"num_write_kb":45747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55631872,"data_stored":81064344,"data_compressed":7543912,"data_compressed_allocated":11722752,"data_compressed_original":37216256,"omap_allocated":0,"internal_metadata":0},"log_size":9557,"ondisk_log_size":9557,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64798,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575199,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61912,"kb_used_data":16848,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94309928,"statfs":{"total":96636764160,"available":96573366272,"internally_reserved":0,"allocated":17252352,"data_stored":25871404,"data_compressed":2449552,"data_compressed_allocated":3768320,"data_compressed_original":12558336,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575199,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61012,"kb_used_data":11788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310828,"statfs":{"total":96636764160,"available":96574287872,"internally_reserved":0,"allocated":12070912,"data_stored":16892518,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575199,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":114720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575199,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60916,"kb_used_data":7532,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310924,"statfs":{"total":96636764160,"available":96574386176,"internally_reserved":0,"allocated":7712768,"data_stored":11064847,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575199,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58952,"kb_used_data":9728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312888,"statfs":{"total":96636764160,"available":96576397312,"internally_reserved":0,"allocated":9961472,"data_stored":12918697,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575199,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":495616,"data_stored":335113,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575199,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47824,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324016,"statfs":{"total":96636764160,"available":96587792384,"internally_reserved":0,"allocated":2826240,"data_stored":4309094,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575199,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60600,"kb_used_data":7216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311240,"statfs":{"total":96636764160,"available":96574709760,"internally_reserved":0,"allocated":7389184,"data_stored":11655727,"data_compressed":1158786,"data_compressed_allocated":1720320,"data_compressed_original":6168576,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32399,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7045120,"data_stored":10950729,"data_compressed":1126387,"data_compressed_allocated":1675264,"data_compressed_original":5595136,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2547712,"data_stored":4194464,"data_compressed":504538,"data_compressed_allocated":720896,"data_compressed_original":2375680,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":220483,"data_compressed":1409,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9682944,"data_stored":12804067,"data_compressed":971477,"data_compressed_allocated":1617920,"data_compressed_original":4747264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7434240,"data_stored":10950217,"data_compressed":1097656,"data_compressed_allocated":1728512,"data_compressed_original":5259264,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11792384,"data_stored":16777888,"data_compressed":1425292,"data_compressed_allocated":2252800,"data_compressed_original":7245824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16908288,"data_stored":25166406,"data_compressed":2417153,"data_compressed_allocated":3723264,"data_compressed_original":11984896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-21T23:54:01.625 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-02-21T23:54:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.763+0000 7f210b6de640 1 Processor -- start 2024-02-21T23:54:01.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.763+0000 7f210b6de640 1 -- start start 2024-02-21T23:54:01.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.764+0000 7f210b6de640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f21040c9450 0x7f2104072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.764+0000 7f210b6de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c9830 0x7f2104150940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.764+0000 7f210b6de640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2104072fa0 con 0x7f21040c7790 2024-02-21T23:54:01.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.764+0000 7f210b6de640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2104073210 con 0x7f21040c9450 2024-02-21T23:54:01.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.764+0000 7f210b6de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210415a280 con 0x7f21040c9830 2024-02-21T23:54:01.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.764+0000 7f210947a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f21040c9450 0x7f2104072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.765+0000 7f2109c7b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c9830 0x7f2104150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.765+0000 7f2109c7b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c9830 0x7f2104150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:41228/0 (socket says 172.21.15.133:41228) 2024-02-21T23:54:01.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.765+0000 7f2109c7b640 1 -- 172.21.15.133:0/4192705975 learned_addr learned my addr 172.21.15.133:0/4192705975 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:01.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.767+0000 7f2109c7b640 1 -- 172.21.15.133:0/4192705975 >> v2:172.21.15.133:3300/0 conn(0x7f21040c9450 msgr2=0x7f2104072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:01.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.767+0000 7f2109c7b640 1 --2- 172.21.15.133:0/4192705975 >> v2:172.21.15.133:3300/0 conn(0x7f21040c9450 0x7f2104072a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.767+0000 7f2109c7b640 1 -- 172.21.15.133:0/4192705975 >> v1:172.21.15.133:6789/0 conn(0x7f21040c7790 legacy=0x7f210415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.767+0000 7f2109c7b640 1 -- 172.21.15.133:0/4192705975 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210415a560 con 0x7f21040c9830 2024-02-21T23:54:01.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.769+0000 7f2109c7b640 1 --2- 172.21.15.133:0/4192705975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c9830 0x7f2104150940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009e50 tx=0x7f20f402bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee410fbc2a05bc8 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:01.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f2103fff640 1 -- 172.21.15.133:0/4192705975 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f402cb90 con 0x7f21040c9830 2024-02-21T23:54:01.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f2103fff640 1 -- 172.21.15.133:0/4192705975 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f402cd30 con 0x7f21040c9830 2024-02-21T23:54:01.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f2103fff640 1 -- 172.21.15.133:0/4192705975 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4034610 con 0x7f21040c9830 2024-02-21T23:54:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f210b6de640 1 -- 172.21.15.133:0/4192705975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c9830 msgr2=0x7f2104150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f210b6de640 1 --2- 172.21.15.133:0/4192705975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c9830 0x7f2104150940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009e50 tx=0x7f20f402bff0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f210b6de640 1 -- 172.21.15.133:0/4192705975 shutdown_connections 2024-02-21T23:54:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f210b6de640 1 --2- 172.21.15.133:0/4192705975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c9830 0x7f2104150940 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f210b6de640 1 --2- 172.21.15.133:0/4192705975 >> v2:172.21.15.133:3300/0 conn(0x7f21040c9450 0x7f2104072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f210b6de640 1 -- 172.21.15.133:0/4192705975 >> 172.21.15.133:0/4192705975 conn(0x7f210406bfd0 msgr2=0x7f210406c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f210b6de640 1 -- 172.21.15.133:0/4192705975 shutdown_connections 2024-02-21T23:54:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.775+0000 7f210b6de640 1 -- 172.21.15.133:0/4192705975 wait complete. 2024-02-21T23:54:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.776+0000 7f210b6de640 1 Processor -- start 2024-02-21T23:54:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.776+0000 7f210b6de640 1 -- start start 2024-02-21T23:54:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.776+0000 7f210b6de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c7790 0x7f21040cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.776+0000 7f210b6de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21040c9450 0x7f210414b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.776+0000 7f210b6de640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040c9830 0x7f21041495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.776+0000 7f210b6de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210415de70 con 0x7f21040c9450 2024-02-21T23:54:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.776+0000 7f210b6de640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210415ab20 con 0x7f21040c9830 2024-02-21T23:54:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.776+0000 7f210b6de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210415dff0 con 0x7f21040c7790 2024-02-21T23:54:01.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.777+0000 7f2108c79640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21040c9450 0x7f210414b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.777+0000 7f2109c7b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040c9830 0x7f21041495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.777+0000 7f210947a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c7790 0x7f21040cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:01.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.777+0000 7f2109c7b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040c9830 0x7f21041495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:54272/0 (socket says 172.21.15.133:54272) 2024-02-21T23:54:01.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.777+0000 7f210947a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c7790 0x7f21040cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:41240/0 (socket says 172.21.15.133:41240) 2024-02-21T23:54:01.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.777+0000 7f2109c7b640 1 -- 172.21.15.133:0/3811431492 learned_addr learned my addr 172.21.15.133:0/3811431492 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:01.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.782+0000 7f2109c7b640 1 -- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c7790 msgr2=0x7f21040cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.782+0000 7f2109c7b640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c7790 0x7f21040cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.782+0000 7f2109c7b640 1 -- 172.21.15.133:0/3811431492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21040c9450 msgr2=0x7f210414b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:01.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.782+0000 7f2109c7b640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21040c9450 0x7f210414b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:01.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.782+0000 7f2109c7b640 1 -- 172.21.15.133:0/3811431492 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21041e4f40 con 0x7f21040c9830 2024-02-21T23:54:01.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.784+0000 7f2109c7b640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040c9830 0x7f21041495a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20f4036040 tx=0x7f20f4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.788+0000 7f2101ffb640 1 -- 172.21.15.133:0/3811431492 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4043020 con 0x7f21040c9830 2024-02-21T23:54:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.788+0000 7f2101ffb640 1 -- 172.21.15.133:0/3811431492 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f403e450 con 0x7f21040c9830 2024-02-21T23:54:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.788+0000 7f2101ffb640 1 -- 172.21.15.133:0/3811431492 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4009510 con 0x7f21040c9830 2024-02-21T23:54:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.788+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21041e47e0 con 0x7f21040c9830 2024-02-21T23:54:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.788+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21041f2d90 con 0x7f21040c9830 2024-02-21T23:54:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:01.788+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210406a130 con 0x7f21040c9830 2024-02-21T23:54:02.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.077+0000 7f2101ffb640 1 -- 172.21.15.133:0/3811431492 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20f4046020 con 0x7f21040c9830 2024-02-21T23:54:02.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.077+0000 7f2101ffb640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f20f006b550 0x7f20f006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:02.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.077+0000 7f210947a640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f20f006b550 0x7f20f006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:02.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.078+0000 7f210947a640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f20f006b550 0x7f20f006da10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009aa0 tx=0x7f20fc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:02.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.087+0000 7f2101ffb640 1 -- 172.21.15.133:0/3811431492 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f20f4079490 con 0x7f21040c9830 2024-02-21T23:54:02.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.103+0000 7f2101ffb640 1 -- 172.21.15.133:0/3811431492 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f20f404d070 con 0x7f21040c9830 2024-02-21T23:54:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.275+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f21040c85e0 con 0x7f21040c9830 2024-02-21T23:54:02.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.505+0000 7f2101ffb640 1 -- 172.21.15.133:0/3811431492 <== mon.1 v2:172.21.15.142:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f20f407b3f0 con 0x7f21040c9830 2024-02-21T23:54:02.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.505+0000 7f2101ffb640 1 -- 172.21.15.133:0/3811431492 <== mon.1 v2:172.21.15.142:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f20f407f490 con 0x7f21040c9830 2024-02-21T23:54:02.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f20f006b550 msgr2=0x7f20f006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:02.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f20f006b550 0x7f20f006da10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009aa0 tx=0x7f20fc009190 comp rx=0 tx=0).stop 2024-02-21T23:54:02.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040c9830 msgr2=0x7f21041495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:02.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040c9830 0x7f21041495a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20f4036040 tx=0x7f20f4004a40 comp rx=0 tx=0).stop 2024-02-21T23:54:02.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 shutdown_connections 2024-02-21T23:54:02.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f20f006b550 0x7f20f006da10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21040c9830 0x7f21041495a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21040c9450 0x7f210414b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 --2- 172.21.15.133:0/3811431492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21040c7790 0x7f21040cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 >> 172.21.15.133:0/3811431492 conn(0x7f210406bfd0 msgr2=0x7f21040cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 shutdown_connections 2024-02-21T23:54:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.506+0000 7f210b6de640 1 -- 172.21.15.133:0/3811431492 wait complete. 2024-02-21T23:54:02.514 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-21T23:54:02.514 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:54:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.620+0000 7fed7a688640 1 Processor -- start 2024-02-21T23:54:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.620+0000 7fed7a688640 1 -- start start 2024-02-21T23:54:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.621+0000 7fed7a688640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 0x7fed7415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.621+0000 7fed7a688640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fed7415b020 0x7fed740cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.621+0000 7fed7a688640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed740cbb30 0x7fed740c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.621+0000 7fed7a688640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed740c2b40 con 0x7fed740cbb30 2024-02-21T23:54:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.621+0000 7fed7a688640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed740c2d70 con 0x7fed7415b020 2024-02-21T23:54:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.621+0000 7fed7a688640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed740c3050 con 0x7fed74150f80 2024-02-21T23:54:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.621+0000 7fed737fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fed7415b020 0x7fed740cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.621+0000 7fed78c25640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed740cbb30 0x7fed740c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.621+0000 7fed73fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 0x7fed7415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.622+0000 7fed737fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fed7415b020 0x7fed740cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:54284/0 (socket says 172.21.15.133:54284) 2024-02-21T23:54:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.622+0000 7fed737fe640 1 -- 172.21.15.133:0/1447217927 learned_addr learned my addr 172.21.15.133:0/1447217927 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.624+0000 7fed737fe640 1 -- 172.21.15.133:0/1447217927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 msgr2=0x7fed7415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.624+0000 7fed737fe640 1 --2- 172.21.15.133:0/1447217927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 0x7fed7415aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.624+0000 7fed737fe640 1 -- 172.21.15.133:0/1447217927 >> v2:172.21.15.133:3300/0 conn(0x7fed740cbb30 msgr2=0x7fed740c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.624+0000 7fed737fe640 1 --2- 172.21.15.133:0/1447217927 >> v2:172.21.15.133:3300/0 conn(0x7fed740cbb30 0x7fed740c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.624+0000 7fed737fe640 1 -- 172.21.15.133:0/1447217927 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed740c3330 con 0x7fed7415b020 2024-02-21T23:54:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.626+0000 7fed737fe640 1 --2- 172.21.15.133:0/1447217927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 0x7fed740cb5f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed7406bd30 tx=0x7fed6802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27b0de3afad91a11 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.628+0000 7fed72ffd640 1 -- 172.21.15.133:0/1447217927 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68034070 con 0x7fed7415b020 2024-02-21T23:54:02.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed72ffd640 1 -- 172.21.15.133:0/1447217927 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fed68030860 con 0x7fed7415b020 2024-02-21T23:54:02.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed72ffd640 1 -- 172.21.15.133:0/1447217927 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68030c50 con 0x7fed7415b020 2024-02-21T23:54:02.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed7a688640 1 -- 172.21.15.133:0/1447217927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 msgr2=0x7fed740cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:02.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed7a688640 1 --2- 172.21.15.133:0/1447217927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 0x7fed740cb5f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed7406bd30 tx=0x7fed6802b9a0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed7a688640 1 -- 172.21.15.133:0/1447217927 shutdown_connections 2024-02-21T23:54:02.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed7a688640 1 --2- 172.21.15.133:0/1447217927 >> v2:172.21.15.133:3300/0 conn(0x7fed740cbb30 0x7fed740c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed7a688640 1 --2- 172.21.15.133:0/1447217927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 0x7fed740cb5f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed7a688640 1 --2- 172.21.15.133:0/1447217927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 0x7fed7415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed7a688640 1 -- 172.21.15.133:0/1447217927 >> 172.21.15.133:0/1447217927 conn(0x7fed7406aec0 msgr2=0x7fed7406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:02.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed7a688640 1 -- 172.21.15.133:0/1447217927 shutdown_connections 2024-02-21T23:54:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.630+0000 7fed7a688640 1 -- 172.21.15.133:0/1447217927 wait complete. 2024-02-21T23:54:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.631+0000 7fed7a688640 1 Processor -- start 2024-02-21T23:54:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.631+0000 7fed7a688640 1 -- start start 2024-02-21T23:54:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.632+0000 7fed7a688640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed740cbb30 0x7fed74147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.632+0000 7fed7a688640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 0x7fed741e1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.632+0000 7fed7a688640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 0x7fed741e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.632+0000 7fed7a688640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed740c5990 con 0x7fed740cbb30 2024-02-21T23:54:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.632+0000 7fed7a688640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7415a5c0 con 0x7fed7415b020 2024-02-21T23:54:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.632+0000 7fed7a688640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed740c5b10 con 0x7fed74150f80 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.632+0000 7fed78c25640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 0x7fed741e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.633+0000 7fed73fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed740cbb30 0x7fed74147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.633+0000 7fed737fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 0x7fed741e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.633+0000 7fed78c25640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 0x7fed741e1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:54286/0 (socket says 172.21.15.133:54286) 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.633+0000 7fed78c25640 1 -- 172.21.15.133:0/1086739329 learned_addr learned my addr 172.21.15.133:0/1086739329 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.636+0000 7fed78c25640 1 -- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 msgr2=0x7fed741e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.636+0000 7fed78c25640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 0x7fed741e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.636+0000 7fed78c25640 1 -- 172.21.15.133:0/1086739329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed740cbb30 msgr2=0x7fed74147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.636+0000 7fed78c25640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed740cbb30 0x7fed74147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.636+0000 7fed78c25640 1 -- 172.21.15.133:0/1086739329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed74147970 con 0x7fed7415b020 2024-02-21T23:54:02.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.638+0000 7fed78c25640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 0x7fed741e1b40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00ef30 tx=0x7fed6c00b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.643+0000 7fed70ff9640 1 -- 172.21.15.133:0/1086739329 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c018020 con 0x7fed7415b020 2024-02-21T23:54:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.643+0000 7fed70ff9640 1 -- 172.21.15.133:0/1086739329 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fed6c014400 con 0x7fed7415b020 2024-02-21T23:54:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.643+0000 7fed70ff9640 1 -- 172.21.15.133:0/1086739329 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0147f0 con 0x7fed7415b020 2024-02-21T23:54:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.643+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed741ef7b0 con 0x7fed7415b020 2024-02-21T23:54:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.643+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed741e0360 con 0x7fed7415b020 2024-02-21T23:54:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.643+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed740c4660 con 0x7fed7415b020 2024-02-21T23:54:02.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.662+0000 7fed70ff9640 1 -- 172.21.15.133:0/1086739329 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed6c005d10 con 0x7fed7415b020 2024-02-21T23:54:02.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.663+0000 7fed70ff9640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fed6006b4d0 0x7fed6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:02.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.663+0000 7fed73fff640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fed6006b4d0 0x7fed6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:02.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.664+0000 7fed73fff640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fed6006b4d0 0x7fed6006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed64009b40 tx=0x7fed64009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:02.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.666+0000 7fed70ff9640 1 -- 172.21.15.133:0/1086739329 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fed6c057800 con 0x7fed7415b020 2024-02-21T23:54:02.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:02.915+0000 7fed70ff9640 1 -- 172.21.15.133:0/1086739329 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fed740c4660 con 0x7fed7415b020 2024-02-21T23:54:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.140+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed740c4310 con 0x7fed7415b020 2024-02-21T23:54:03.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.162+0000 7fed70ff9640 1 -- 172.21.15.133:0/1086739329 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6535 (secure 0 0 0) 0x7fed740c4310 con 0x7fed7415b020 2024-02-21T23:54:03.157 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:54:03.157 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1319340142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1319340142},{"type":"v1","addr":"172.21.15.133:6837","nonce":1319340142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1740931674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1740931674},{"type":"v1","addr":"172.21.15.142:6837","nonce":1740931674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3712864993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3712864993},{"type":"v1","addr":"172.21.15.142:6839","nonce":3712864993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2928905924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2928905924},{"type":"v1","addr":"172.21.15.133:6833","nonce":2928905924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3461010406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3461010406},{"type":"v1","addr":"172.21.15.133:6835","nonce":3461010406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-21T23:25:16.775942+0000","modified":"2024-02-21T23:25:49.141553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6835/1527124199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1527124199},{"type":"v1","addr":"172.21.15.142:6835","nonce":1527124199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-21T23:54:03.157 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2024-02-21T23:54:03.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fed6006b4d0 msgr2=0x7fed6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fed6006b4d0 0x7fed6006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed64009b40 tx=0x7fed64009190 comp rx=0 tx=0).stop 2024-02-21T23:54:03.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 msgr2=0x7fed741e1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 0x7fed741e1b40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00ef30 tx=0x7fed6c00b480 comp rx=0 tx=0).stop 2024-02-21T23:54:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 shutdown_connections 2024-02-21T23:54:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fed6006b4d0 0x7fed6006d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed7415b020 0x7fed741e1b40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed74150f80 0x7fed741e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 --2- 172.21.15.133:0/1086739329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed740cbb30 0x7fed74147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 >> 172.21.15.133:0/1086739329 conn(0x7fed7406aec0 msgr2=0x7fed741597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 shutdown_connections 2024-02-21T23:54:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.165+0000 7fed7a688640 1 -- 172.21.15.133:0/1086739329 wait complete. 2024-02-21T23:54:03.171 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-21T23:54:03.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.278+0000 7fef35a36640 1 Processor -- start 2024-02-21T23:54:03.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.278+0000 7fef35a36640 1 -- start start 2024-02-21T23:54:03.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.279+0000 7fef35a36640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef30154490 0x7fef300ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.279+0000 7fef35a36640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fef300c6cc0 0x7fef300cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.279+0000 7fef35a36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef300cbfc0 0x7fef300cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.279+0000 7fef35a36640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef30154870 con 0x7fef300c6cc0 2024-02-21T23:54:03.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.279+0000 7fef35a36640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3015a0c0 con 0x7fef30154490 2024-02-21T23:54:03.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.279+0000 7fef35a36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef300d03e0 con 0x7fef300cbfc0 2024-02-21T23:54:03.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.279+0000 7fef2e7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fef300c6cc0 0x7fef300cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.280+0000 7fef2f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef300cbfc0 0x7fef300cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.280+0000 7fef2e7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fef300c6cc0 0x7fef300cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45828/0 (socket says 172.21.15.133:45828) 2024-02-21T23:54:03.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.280+0000 7fef2e7fc640 1 -- 172.21.15.133:0/1591327611 learned_addr learned my addr 172.21.15.133:0/1591327611 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:03.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.282+0000 7fef2effd640 1 --2- 172.21.15.133:0/1591327611 >> v2:172.21.15.142:3300/0 conn(0x7fef30154490 0x7fef300ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.282+0000 7fef2e7fc640 1 -- 172.21.15.133:0/1591327611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef300cbfc0 msgr2=0x7fef300cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:03.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.282+0000 7fef2e7fc640 1 --2- 172.21.15.133:0/1591327611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef300cbfc0 0x7fef300cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.282+0000 7fef2e7fc640 1 -- 172.21.15.133:0/1591327611 >> v2:172.21.15.142:3300/0 conn(0x7fef30154490 msgr2=0x7fef300ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:03.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.282+0000 7fef2e7fc640 1 --2- 172.21.15.133:0/1591327611 >> v2:172.21.15.142:3300/0 conn(0x7fef30154490 0x7fef300ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.282+0000 7fef2e7fc640 1 -- 172.21.15.133:0/1591327611 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3015a380 con 0x7fef300c6cc0 2024-02-21T23:54:03.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.284+0000 7fef2e7fc640 1 --2- 172.21.15.133:0/1591327611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300c6cc0 0x7fef300cac60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef3006c810 tx=0x7fef2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff10af3344b4b7f server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:03.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef2dffb640 1 -- 172.21.15.133:0/1591327611 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2402d040 con 0x7fef300c6cc0 2024-02-21T23:54:03.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef2dffb640 1 -- 172.21.15.133:0/1591327611 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fef24004230 con 0x7fef300c6cc0 2024-02-21T23:54:03.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef2dffb640 1 -- 172.21.15.133:0/1591327611 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24003610 con 0x7fef300c6cc0 2024-02-21T23:54:03.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef35a36640 1 -- 172.21.15.133:0/1591327611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300c6cc0 msgr2=0x7fef300cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef35a36640 1 --2- 172.21.15.133:0/1591327611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300c6cc0 0x7fef300cac60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef3006c810 tx=0x7fef2402b8f0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef35a36640 1 -- 172.21.15.133:0/1591327611 shutdown_connections 2024-02-21T23:54:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef35a36640 1 --2- 172.21.15.133:0/1591327611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef300cbfc0 0x7fef300cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef35a36640 1 --2- 172.21.15.133:0/1591327611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300c6cc0 0x7fef300cac60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef35a36640 1 --2- 172.21.15.133:0/1591327611 >> v2:172.21.15.142:3300/0 conn(0x7fef30154490 0x7fef300ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.289+0000 7fef35a36640 1 -- 172.21.15.133:0/1591327611 >> 172.21.15.133:0/1591327611 conn(0x7fef30072800 msgr2=0x7fef3006be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.290+0000 7fef35a36640 1 -- 172.21.15.133:0/1591327611 shutdown_connections 2024-02-21T23:54:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.290+0000 7fef35a36640 1 -- 172.21.15.133:0/1591327611 wait complete. 2024-02-21T23:54:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.291+0000 7fef35a36640 1 Processor -- start 2024-02-21T23:54:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.291+0000 7fef35a36640 1 -- start start 2024-02-21T23:54:03.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.292+0000 7fef35a36640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef300c6cc0 0x7fef3015d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.292+0000 7fef35a36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300cbfc0 0x7fef301e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.292+0000 7fef35a36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef30154490 0x7fef301e5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.292+0000 7fef35a36640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3015ac20 con 0x7fef300cbfc0 2024-02-21T23:54:03.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.292+0000 7fef35a36640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3015a940 con 0x7fef300c6cc0 2024-02-21T23:54:03.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.292+0000 7fef35a36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef300cf850 con 0x7fef30154490 2024-02-21T23:54:03.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.292+0000 7fef2e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300cbfc0 0x7fef301e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.293+0000 7fef2effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef300c6cc0 0x7fef3015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.293+0000 7fef2f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef30154490 0x7fef301e5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.293+0000 7fef2e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300cbfc0 0x7fef301e57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45838/0 (socket says 172.21.15.133:45838) 2024-02-21T23:54:03.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.293+0000 7fef2e7fc640 1 -- 172.21.15.133:0/3541271397 learned_addr learned my addr 172.21.15.133:0/3541271397 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:03.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.296+0000 7fef2e7fc640 1 -- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef30154490 msgr2=0x7fef301e5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.296+0000 7fef2e7fc640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef30154490 0x7fef301e5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.296+0000 7fef2e7fc640 1 -- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef300c6cc0 msgr2=0x7fef3015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.296+0000 7fef2e7fc640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef300c6cc0 0x7fef3015d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.296+0000 7fef2e7fc640 1 -- 172.21.15.133:0/3541271397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef301e7590 con 0x7fef300cbfc0 2024-02-21T23:54:03.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.298+0000 7fef2e7fc640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300cbfc0 0x7fef301e57e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef2402b930 tx=0x7fef240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.302+0000 7fef17fff640 1 -- 172.21.15.133:0/3541271397 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24040070 con 0x7fef300cbfc0 2024-02-21T23:54:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.302+0000 7fef17fff640 1 -- 172.21.15.133:0/3541271397 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fef24031850 con 0x7fef300cbfc0 2024-02-21T23:54:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.302+0000 7fef17fff640 1 -- 172.21.15.133:0/3541271397 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24031c00 con 0x7fef300cbfc0 2024-02-21T23:54:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.302+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef301e6ff0 con 0x7fef300cbfc0 2024-02-21T23:54:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.302+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef301e4fe0 con 0x7fef300cbfc0 2024-02-21T23:54:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.303+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3006a130 con 0x7fef300cbfc0 2024-02-21T23:54:03.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.320+0000 7fef17fff640 1 -- 172.21.15.133:0/3541271397 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef240040f0 con 0x7fef300cbfc0 2024-02-21T23:54:03.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.321+0000 7fef17fff640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fef1c06b520 0x7fef1c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.321+0000 7fef2effd640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fef1c06b520 0x7fef1c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.321+0000 7fef2effd640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fef1c06b520 0x7fef1c06d9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef2000aa20 tx=0x7fef20008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:03.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.324+0000 7fef17fff640 1 -- 172.21.15.133:0/3541271397 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fef24078d50 con 0x7fef300cbfc0 2024-02-21T23:54:03.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.560+0000 7fef17fff640 1 -- 172.21.15.133:0/3541271397 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fef2407acb0 con 0x7fef300cbfc0 2024-02-21T23:54:03.576 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:54:03.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:54:03.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.782+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef300cbda0 con 0x7fef300cbfc0 2024-02-21T23:54:03.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.795+0000 7fef17fff640 1 -- 172.21.15.133:0/3541271397 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6535 (secure 0 0 0) 0x7fef2407ed50 con 0x7fef300cbfc0 2024-02-21T23:54:03.790 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:54:03.790 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1319340142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1319340142},{"type":"v1","addr":"172.21.15.133:6837","nonce":1319340142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1740931674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1740931674},{"type":"v1","addr":"172.21.15.142:6837","nonce":1740931674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3712864993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3712864993},{"type":"v1","addr":"172.21.15.142:6839","nonce":3712864993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2928905924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2928905924},{"type":"v1","addr":"172.21.15.133:6833","nonce":2928905924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3461010406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3461010406},{"type":"v1","addr":"172.21.15.133:6835","nonce":3461010406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-21T23:25:16.775942+0000","modified":"2024-02-21T23:25:49.141553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6835/1527124199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1527124199},{"type":"v1","addr":"172.21.15.142:6835","nonce":1527124199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-21T23:54:03.790 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fef1c06b520 msgr2=0x7fef1c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fef1c06b520 0x7fef1c06d9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef2000aa20 tx=0x7fef20008040 comp rx=0 tx=0).stop 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300cbfc0 msgr2=0x7fef301e57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300cbfc0 0x7fef301e57e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef2402b930 tx=0x7fef240046c0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 shutdown_connections 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fef1c06b520 0x7fef1c06d9e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef30154490 0x7fef301e5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef300cbfc0 0x7fef301e57e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 --2- 172.21.15.133:0/3541271397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef300c6cc0 0x7fef3015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 >> 172.21.15.133:0/3541271397 conn(0x7fef30072800 msgr2=0x7fef300cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:03.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 shutdown_connections 2024-02-21T23:54:03.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.797+0000 7fef35a36640 1 -- 172.21.15.133:0/3541271397 wait complete. 2024-02-21T23:54:03.804 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-21T23:54:03.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.909+0000 7fdf1349d640 1 Processor -- start 2024-02-21T23:54:03.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.910+0000 7fdf1349d640 1 -- start start 2024-02-21T23:54:03.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.910+0000 7fdf1349d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdf0c0cdfe0 0x7fdf0c0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.910+0000 7fdf1349d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0717e0 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.910+0000 7fdf1349d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c158090 con 0x7fdf0c14e510 2024-02-21T23:54:03.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.910+0000 7fdf1349d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c0c84b0 con 0x7fdf0c0cdfe0 2024-02-21T23:54:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.911+0000 7fdf11239640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdf0c0cdfe0 0x7fdf0c0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.911+0000 7fdf11a3a640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fdf0c14e510 0x7fdf0c14e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:56444/0 (socket says 172.21.15.133:56444) 2024-02-21T23:54:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.911+0000 7fdf11a3a640 1 -- 172.21.15.133:0/2767514355 learned_addr learned my addr 172.21.15.133:0/2767514355 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:54:03.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.913+0000 7fdf03fff640 1 -- 172.21.15.133:0/2767514355 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572323269 0 0) 0x7fdf0c158090 con 0x7fdf0c14e510 2024-02-21T23:54:03.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.913+0000 7fdf03fff640 1 -- 172.21.15.133:0/2767514355 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef8003520 con 0x7fdf0c14e510 2024-02-21T23:54:03.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.914+0000 7fdf11239640 1 -- 172.21.15.133:0/2767514355 >> v1:172.21.15.142:6789/0 conn(0x7fdf0c14e510 legacy=0x7fdf0c14e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.914+0000 7fdf11239640 1 -- 172.21.15.133:0/2767514355 >> v1:172.21.15.133:6789/0 conn(0x7fdf0c0ce3c0 legacy=0x7fdf0c070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.914+0000 7fdf11239640 1 -- 172.21.15.133:0/2767514355 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c158370 con 0x7fdf0c0cdfe0 2024-02-21T23:54:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.918+0000 7fdf11239640 1 --2- 172.21.15.133:0/2767514355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 0x7fdf0c0c3250 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdf080097a0 tx=0x7fdf0802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6e93883975dbb9 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.922+0000 7fdf03fff640 1 -- 172.21.15.133:0/2767514355 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08034070 con 0x7fdf0c0cdfe0 2024-02-21T23:54:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.922+0000 7fdf03fff640 1 -- 172.21.15.133:0/2767514355 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdf08030610 con 0x7fdf0c0cdfe0 2024-02-21T23:54:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.922+0000 7fdf03fff640 1 -- 172.21.15.133:0/2767514355 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08030a00 con 0x7fdf0c0cdfe0 2024-02-21T23:54:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.923+0000 7fdf1349d640 1 -- 172.21.15.133:0/2767514355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 msgr2=0x7fdf0c0c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.923+0000 7fdf1349d640 1 --2- 172.21.15.133:0/2767514355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 0x7fdf0c0c3250 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdf080097a0 tx=0x7fdf0802b960 comp rx=0 tx=0).stop 2024-02-21T23:54:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.923+0000 7fdf1349d640 1 -- 172.21.15.133:0/2767514355 shutdown_connections 2024-02-21T23:54:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.923+0000 7fdf1349d640 1 --2- 172.21.15.133:0/2767514355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 0x7fdf0c0c3250 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.923+0000 7fdf1349d640 1 -- 172.21.15.133:0/2767514355 >> 172.21.15.133:0/2767514355 conn(0x7fdf0c06aec0 msgr2=0x7fdf0c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.923+0000 7fdf1349d640 1 -- 172.21.15.133:0/2767514355 shutdown_connections 2024-02-21T23:54:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.923+0000 7fdf1349d640 1 -- 172.21.15.133:0/2767514355 wait complete. 2024-02-21T23:54:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.924+0000 7fdf1349d640 1 Processor -- start 2024-02-21T23:54:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.924+0000 7fdf1349d640 1 -- start start 2024-02-21T23:54:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.925+0000 7fdf1349d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 0x7fdf0c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.925+0000 7fdf1349d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c0ce3c0 0x7fdf0c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.925+0000 7fdf1349d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c14e510 0x7fdf0c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.925+0000 7fdf1349d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c158930 con 0x7fdf0c0cdfe0 2024-02-21T23:54:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.925+0000 7fdf1349d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c158ca0 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.925+0000 7fdf1349d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c071480 con 0x7fdf0c14e510 2024-02-21T23:54:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.925+0000 7fdf11239640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 0x7fdf0c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.925+0000 7fdf10a38640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c0ce3c0 0x7fdf0c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.925+0000 7fdf11a3a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c14e510 0x7fdf0c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.926+0000 7fdf11239640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 0x7fdf0c0c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45856/0 (socket says 172.21.15.133:45856) 2024-02-21T23:54:03.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.926+0000 7fdf11239640 1 -- 172.21.15.133:0/2959112041 learned_addr learned my addr 172.21.15.133:0/2959112041 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:03.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.929+0000 7fdf10a38640 1 -- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c14e510 msgr2=0x7fdf0c06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.929+0000 7fdf10a38640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c14e510 0x7fdf0c06d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.929+0000 7fdf10a38640 1 -- 172.21.15.133:0/2959112041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 msgr2=0x7fdf0c0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:03.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.929+0000 7fdf10a38640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 0x7fdf0c0c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:03.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.929+0000 7fdf10a38640 1 -- 172.21.15.133:0/2959112041 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0b1ee0 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.931+0000 7fdf10a38640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c0ce3c0 0x7fdf0c0b2470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c340 tx=0x7fdefc00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:03.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.935+0000 7fdf01ffb640 1 -- 172.21.15.133:0/2959112041 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0052d0 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.935+0000 7fdf01ffb640 1 -- 172.21.15.133:0/2959112041 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdefc005490 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.935+0000 7fdf01ffb640 1 -- 172.21.15.133:0/2959112041 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc007280 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.935+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c0ca660 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.935+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c0c8f10 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.936+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c06a130 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.955+0000 7fdf01ffb640 1 -- 172.21.15.133:0/2959112041 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdefc007420 con 0x7fdf0c0ce3c0 2024-02-21T23:54:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.955+0000 7fdf01ffb640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fdef006b520 0x7fdef006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:03.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.955+0000 7fdf11239640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fdef006b520 0x7fdef006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:03.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.956+0000 7fdf11239640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fdef006b520 0x7fdef006d9e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802b9a0 tx=0x7fdf0803b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:03.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:03.958+0000 7fdf01ffb640 1 -- 172.21.15.133:0/2959112041 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fdefc056330 con 0x7fdf0c0ce3c0 2024-02-21T23:54:04.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.205+0000 7fdf01ffb640 1 -- 172.21.15.133:0/2959112041 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fdf0c0b1ee0 con 0x7fdf0c0ce3c0 2024-02-21T23:54:04.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.378+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf0c0b0480 con 0x7fdf0c0ce3c0 2024-02-21T23:54:04.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.418+0000 7fdf01ffb640 1 -- 172.21.15.133:0/2959112041 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+19091 (secure 0 0 0) 0x7fdefc058290 con 0x7fdf0c0ce3c0 2024-02-21T23:54:04.413 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-02-21T23:54:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"1f1c3b6d-6263-48c3-85b7-ccc808770c4d","created":"2024-02-21T23:24:48.871317+0000","modified":"2024-02-21T23:25:57.029812+0000","last_up_change":"2024-02-21T23:25:02.449240+0000","last_in_change":"2024-02-21T23:24:54.046821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-21T23:25:02.902698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-21T23:25:13.993803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-21T23:25:15.604077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-21T23:25:19.468744+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dd65829d-e81e-4ce3-82a8-0232e6ec2d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6801","nonce":2680579490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6803","nonce":2680579490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6807","nonce":2680579490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2680579490},{"type":"v1","addr":"172.21.15.133:6805","nonce":2680579490}]},"public_addr":"172.21.15.133:6801/2680579490","cluster_addr":"172.21.15.133:6803/2680579490","heartbeat_back_addr":"172.21.15.133:6807/2680579490","heartbeat_front_addr":"172.21.15.133:6805/2680579490","state":["exists","up"]},{"osd":1,"uuid":"8b21a887-709e-4de1-8f09-0e0dffd47b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6809","nonce":198853132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6811","nonce":198853132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6815","nonce":198853132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":198853132},{"type":"v1","addr":"172.21.15.133:6813","nonce":198853132}]},"public_addr":"172.21.15.133:6809/198853132","cluster_addr":"172.21.15.133:6811/198853132","heartbeat_back_addr":"172.21.15.133:6815/198853132","heartbeat_front_addr":"172.21.15.133:6813/198853132","state":["exists","up"]},{"osd":2,"uuid":"9afcbb30-e40c-4c41-83c2-6886fcb8ee78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6817","nonce":3824311867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6819","nonce":3824311867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6823","nonce":3824311867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3824311867},{"type":"v1","addr":"172.21.15.133:6821","nonce":3824311867}]},"public_addr":"172.21.15.133:6817/3824311867","cluster_addr":"172.21.15.133:6819/3824311867","heartbeat_back_addr":"172.21.15.133:6823/3824311867","heartbeat_front_addr":"172.21.15.133:6821/3824311867","state":["exists","up"]},{"osd":3,"uuid":"6d376633-e1d0-415c-86b9-1b94a1775497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6825","nonce":4089028447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6827","nonce":4089028447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6831","nonce":4089028447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":4089028447},{"type":"v1","addr":"172.21.15.133:6829","nonce":4089028447}]},"public_addr":"172.21.15.133:6825/4089028447","cluster_addr":"172.21.15.133:6827/4089028447","heartbeat_back_addr":"172.21.15.133:6831/4089028447","heartbeat_front_addr":"172.21.15.133:6829/4089028447","state":["exists","up"]},{"osd":4,"uuid":"a47be052-7790-4954-af9b-20ec83394f00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6827","nonce":2259485122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6829","nonce":2259485122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6833","nonce":2259485122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2259485122},{"type":"v1","addr":"172.21.15.142:6831","nonce":2259485122}]},"public_addr":"172.21.15.142:6827/2259485122","cluster_addr":"172.21.15.142:6829/2259485122","heartbeat_back_addr":"172.21.15.142:6833/2259485122","heartbeat_front_addr":"172.21.15.142:6831/2259485122","state":["exists","up"]},{"osd":5,"uuid":"bf00fa69-437a-4077-8ef5-629955908024","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6819","nonce":3321674168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6821","nonce":3321674168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6825","nonce":3321674168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3321674168},{"type":"v1","addr":"172.21.15.142:6823","nonce":3321674168}]},"public_addr":"172.21.15.142:6819/3321674168","cluster_addr":"172.21.15.142:6821/3321674168","heartbeat_back_addr":"172.21.15.142:6825/3321674168","heartbeat_front_addr":"172.21.15.142:6823/3321674168","state":["exists","up"]},{"osd":6,"uuid":"9ce52b5c-d828-40a1-80bf-3456f148f2a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6803","nonce":4241695382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6805","nonce":4241695382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6809","nonce":4241695382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4241695382},{"type":"v1","addr":"172.21.15.142:6807","nonce":4241695382}]},"public_addr":"172.21.15.142:6803/4241695382","cluster_addr":"172.21.15.142:6805/4241695382","heartbeat_back_addr":"172.21.15.142:6809/4241695382","heartbeat_front_addr":"172.21.15.142:6807/4241695382","state":["exists","up"]},{"osd":7,"uuid":"a3e6bc6d-31fa-4304-ae53-07f0bfa88924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6811","nonce":3083092904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6813","nonce":3083092904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6817","nonce":3083092904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3083092904},{"type":"v1","addr":"172.21.15.142:6815","nonce":3083092904}]},"public_addr":"172.21.15.142:6811/3083092904","cluster_addr":"172.21.15.142:6813/3083092904","heartbeat_back_addr":"172.21.15.142:6817/3083092904","heartbeat_front_addr":"172.21.15.142:6815/3083092904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.175597+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.253138+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.230531+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.262393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.239448+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.270930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.238777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-21T23:24:59.241970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":0}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-21T23:54:04.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fdef006b520 msgr2=0x7fdef006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:04.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fdef006b520 0x7fdef006d9e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802b9a0 tx=0x7fdf0803b040 comp rx=0 tx=0).stop 2024-02-21T23:54:04.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c0ce3c0 msgr2=0x7fdf0c0b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:04.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c0ce3c0 0x7fdf0c0b2470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c340 tx=0x7fdefc00c800 comp rx=0 tx=0).stop 2024-02-21T23:54:04.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 shutdown_connections 2024-02-21T23:54:04.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fdef006b520 0x7fdef006d9e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:04.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c14e510 0x7fdf0c06d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:04.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c0ce3c0 0x7fdf0c0b2470 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:04.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 --2- 172.21.15.133:0/2959112041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf0c0cdfe0 0x7fdf0c0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:04.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 >> 172.21.15.133:0/2959112041 conn(0x7fdf0c06aec0 msgr2=0x7fdf0c0c31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:04.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 shutdown_connections 2024-02-21T23:54:04.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.420+0000 7fdf1349d640 1 -- 172.21.15.133:0/2959112041 wait complete. 2024-02-21T23:54:04.427 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-21T23:54:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.534+0000 7f4803f9c640 1 Processor -- start 2024-02-21T23:54:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.534+0000 7f4803f9c640 1 -- start start 2024-02-21T23:54:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.535+0000 7f4803f9c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f47fc154300 0x7f47fc06d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.535+0000 7f4803f9c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f47fc0b0fe0 0x7f47fc072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.535+0000 7f4803f9c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47fc15afd0 con 0x7f47fc072f80 2024-02-21T23:54:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.535+0000 7f4803f9c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47fc15a170 con 0x7f47fc154300 2024-02-21T23:54:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.535+0000 7f4803f9c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47fc15a450 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.536+0000 7f4801d38640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f47fc154300 0x7f47fc06d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:04.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.536+0000 7f4802539640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f47fc072f80 0x7f47fc06de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:56448/0 (socket says 172.21.15.133:56448) 2024-02-21T23:54:04.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.536+0000 7f4802539640 1 -- 172.21.15.133:0/1176094885 learned_addr learned my addr 172.21.15.133:0/1176094885 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:54:04.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.536+0000 7f4801537640 1 --2- 172.21.15.133:0/1176094885 >> v2:172.21.15.142:3300/0 conn(0x7f47fc0b0fe0 0x7f47fc072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.538+0000 7f4801d38640 1 -- 172.21.15.133:0/1176094885 >> v2:172.21.15.142:3300/0 conn(0x7f47fc0b0fe0 msgr2=0x7f47fc072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.538+0000 7f4801d38640 1 --2- 172.21.15.133:0/1176094885 >> v2:172.21.15.142:3300/0 conn(0x7f47fc0b0fe0 0x7f47fc072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.538+0000 7f4801d38640 1 -- 172.21.15.133:0/1176094885 >> v1:172.21.15.142:6789/0 conn(0x7f47fc072f80 legacy=0x7f47fc06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.538+0000 7f4801d38640 1 -- 172.21.15.133:0/1176094885 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc15a730 con 0x7f47fc154300 2024-02-21T23:54:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.541+0000 7f4801d38640 1 --2- 172.21.15.133:0/1176094885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47fc154300 0x7f47fc06d0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f47fc061970 tx=0x7f47f802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d21cbf59725f1c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.544+0000 7f4800d36640 1 -- 172.21.15.133:0/1176094885 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8039070 con 0x7f47fc154300 2024-02-21T23:54:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.544+0000 7f4800d36640 1 -- 172.21.15.133:0/1176094885 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f47f802cd00 con 0x7f47fc154300 2024-02-21T23:54:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.544+0000 7f4800d36640 1 -- 172.21.15.133:0/1176094885 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8034680 con 0x7f47fc154300 2024-02-21T23:54:04.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.545+0000 7f4803f9c640 1 -- 172.21.15.133:0/1176094885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47fc154300 msgr2=0x7f47fc06d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:04.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.545+0000 7f4803f9c640 1 --2- 172.21.15.133:0/1176094885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47fc154300 0x7f47fc06d0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f47fc061970 tx=0x7f47f802bf50 comp rx=0 tx=0).stop 2024-02-21T23:54:04.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.545+0000 7f4803f9c640 1 -- 172.21.15.133:0/1176094885 shutdown_connections 2024-02-21T23:54:04.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.545+0000 7f4803f9c640 1 --2- 172.21.15.133:0/1176094885 >> v2:172.21.15.142:3300/0 conn(0x7f47fc0b0fe0 0x7f47fc072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:04.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.545+0000 7f4803f9c640 1 --2- 172.21.15.133:0/1176094885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47fc154300 0x7f47fc06d0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:04.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.545+0000 7f4803f9c640 1 -- 172.21.15.133:0/1176094885 >> 172.21.15.133:0/1176094885 conn(0x7f47fc0cc620 msgr2=0x7f47fc0c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:04.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.545+0000 7f4803f9c640 1 -- 172.21.15.133:0/1176094885 shutdown_connections 2024-02-21T23:54:04.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.545+0000 7f4803f9c640 1 -- 172.21.15.133:0/1176094885 wait complete. 2024-02-21T23:54:04.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.547+0000 7f4803f9c640 1 Processor -- start 2024-02-21T23:54:04.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.547+0000 7f4803f9c640 1 -- start start 2024-02-21T23:54:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.547+0000 7f4803f9c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47fc072f80 0x7f47fc15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.547+0000 7f4803f9c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47fc0b0fe0 0x7f47fc1f1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.547+0000 7f4803f9c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47fc154300 0x7f47fc1f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.547+0000 7f4803f9c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc1546e0 con 0x7f47fc154300 2024-02-21T23:54:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.547+0000 7f4803f9c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc15acf0 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.547+0000 7f4803f9c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc06d5e0 con 0x7f47fc072f80 2024-02-21T23:54:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.548+0000 7f4802539640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47fc154300 0x7f47fc1f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.548+0000 7f4801537640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47fc0b0fe0 0x7f47fc1f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.548+0000 7f4801d38640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47fc072f80 0x7f47fc15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.548+0000 7f4801537640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47fc0b0fe0 0x7f47fc1f1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:54352/0 (socket says 172.21.15.133:54352) 2024-02-21T23:54:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.548+0000 7f4801537640 1 -- 172.21.15.133:0/1611683013 learned_addr learned my addr 172.21.15.133:0/1611683013 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:04.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.551+0000 7f4801537640 1 -- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47fc072f80 msgr2=0x7f47fc15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:04.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.551+0000 7f4801537640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47fc072f80 0x7f47fc15d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:04.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.551+0000 7f4801537640 1 -- 172.21.15.133:0/1611683013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47fc154300 msgr2=0x7f47fc1f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:04.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.551+0000 7f4801537640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47fc154300 0x7f47fc1f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:04.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.551+0000 7f4801537640 1 -- 172.21.15.133:0/1611683013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc15b480 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.554+0000 7f4801537640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47fc0b0fe0 0x7f47fc1f1280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00e9b0 tx=0x7f47ec00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:04.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.556+0000 7f47f27fc640 1 -- 172.21.15.133:0/1611683013 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec017070 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.557+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc15c140 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.557+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc15c6c0 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.557+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47fc06a130 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.558+0000 7f47f27fc640 1 -- 172.21.15.133:0/1611683013 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f47ec00c9a0 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.558+0000 7f47f27fc640 1 -- 172.21.15.133:0/1611683013 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec021060 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.812+0000 7f47f27fc640 1 -- 172.21.15.133:0/1611683013 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47ec01a070 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.812+0000 7f47f27fc640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f47e006b4d0 0x7f47e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:04.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.813+0000 7f4801d38640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f47e006b4d0 0x7f47e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:04.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.813+0000 7f4801d38640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f47e006b4d0 0x7f47e006d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f47f8036000 tx=0x7f47f8030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:04.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.816+0000 7f47f27fc640 1 -- 172.21.15.133:0/1611683013 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f47ec0575a0 con 0x7f47fc0b0fe0 2024-02-21T23:54:04.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:04.826+0000 7f47f27fc640 1 -- 172.21.15.133:0/1611683013 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f47ec0592a0 con 0x7f47fc0b0fe0 2024-02-21T23:54:05.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.047+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47fc06c880 con 0x7f47fc0b0fe0 2024-02-21T23:54:05.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.254+0000 7f47f27fc640 1 -- 172.21.15.133:0/1611683013 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v11) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47ec05d340 con 0x7f47fc0b0fe0 2024-02-21T23:54:05.249 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-21T23:54:05.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.256+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f47e006b4d0 msgr2=0x7f47e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f47e006b4d0 0x7f47e006d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f47f8036000 tx=0x7f47f8030000 comp rx=0 tx=0).stop 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47fc0b0fe0 msgr2=0x7f47fc1f1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47fc0b0fe0 0x7f47fc1f1280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00e9b0 tx=0x7f47ec00ee70 comp rx=0 tx=0).stop 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 shutdown_connections 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f47e006b4d0 0x7f47e006d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47fc154300 0x7f47fc1f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47fc0b0fe0 0x7f47fc1f1280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 --2- 172.21.15.133:0/1611683013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47fc072f80 0x7f47fc15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 >> 172.21.15.133:0/1611683013 conn(0x7f47fc0cc620 msgr2=0x7f47fc157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 shutdown_connections 2024-02-21T23:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.257+0000 7f4803f9c640 1 -- 172.21.15.133:0/1611683013 wait complete. 2024-02-21T23:54:05.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-21T23:54:05.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.371+0000 7f9483b1e640 1 Processor -- start 2024-02-21T23:54:05.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.371+0000 7f9483b1e640 1 -- start start 2024-02-21T23:54:05.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.372+0000 7f9483b1e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f947c0cdfe0 0x7f947c0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:05.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.372+0000 7f9483b1e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f947c0ce3c0 0x7f947c070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:05.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.372+0000 7f9483b1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0712b0 0x7f947c14e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:05.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.372+0000 7f9483b1e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947c0c84b0 con 0x7f947c0ce3c0 2024-02-21T23:54:05.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.372+0000 7f9483b1e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947c158ca0 con 0x7f947c0cdfe0 2024-02-21T23:54:05.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.372+0000 7f9483b1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0717e0 con 0x7f947c0712b0 2024-02-21T23:54:05.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.373+0000 7f94818ba640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f947c0cdfe0 0x7f947c0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:05.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.373+0000 7f94810b9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f947c0ce3c0 0x7f947c070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:05.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.373+0000 7f94820bb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0712b0 0x7f947c14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:05.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.373+0000 7f94818ba640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f947c0cdfe0 0x7f947c0c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:54358/0 (socket says 172.21.15.133:54358) 2024-02-21T23:54:05.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.373+0000 7f94818ba640 1 -- 172.21.15.133:0/3960027140 learned_addr learned my addr 172.21.15.133:0/3960027140 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:05.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.375+0000 7f94818ba640 1 -- 172.21.15.133:0/3960027140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0712b0 msgr2=0x7f947c14e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:05.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.375+0000 7f94818ba640 1 --2- 172.21.15.133:0/3960027140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0712b0 0x7f947c14e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.375+0000 7f94818ba640 1 -- 172.21.15.133:0/3960027140 >> v2:172.21.15.133:3300/0 conn(0x7f947c0ce3c0 msgr2=0x7f947c070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:05.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.375+0000 7f94818ba640 1 --2- 172.21.15.133:0/3960027140 >> v2:172.21.15.133:3300/0 conn(0x7f947c0ce3c0 0x7f947c070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.375+0000 7f94818ba640 1 -- 172.21.15.133:0/3960027140 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c158090 con 0x7f947c0cdfe0 2024-02-21T23:54:05.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.377+0000 7f94818ba640 1 --2- 172.21.15.133:0/3960027140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0cdfe0 0x7f947c0c3250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f947c06b3b0 tx=0x7f947802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca096b0938ba560 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:05.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.381+0000 7f94808b8640 1 -- 172.21.15.133:0/3960027140 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947802ca70 con 0x7f947c0cdfe0 2024-02-21T23:54:05.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.381+0000 7f94808b8640 1 -- 172.21.15.133:0/3960027140 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f947802cc10 con 0x7f947c0cdfe0 2024-02-21T23:54:05.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.381+0000 7f94808b8640 1 -- 172.21.15.133:0/3960027140 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780343f0 con 0x7f947c0cdfe0 2024-02-21T23:54:05.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.382+0000 7f9483b1e640 1 -- 172.21.15.133:0/3960027140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0cdfe0 msgr2=0x7f947c0c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:05.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.382+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3960027140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0cdfe0 0x7f947c0c3250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f947c06b3b0 tx=0x7f947802bed0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.382+0000 7f9483b1e640 1 -- 172.21.15.133:0/3960027140 shutdown_connections 2024-02-21T23:54:05.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.382+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3960027140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0712b0 0x7f947c14e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.382+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3960027140 >> v2:172.21.15.133:3300/0 conn(0x7f947c0ce3c0 0x7f947c070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.382+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3960027140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0cdfe0 0x7f947c0c3250 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.382+0000 7f9483b1e640 1 -- 172.21.15.133:0/3960027140 >> 172.21.15.133:0/3960027140 conn(0x7f947c06aec0 msgr2=0x7f947c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:05.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.382+0000 7f9483b1e640 1 -- 172.21.15.133:0/3960027140 shutdown_connections 2024-02-21T23:54:05.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.382+0000 7f9483b1e640 1 -- 172.21.15.133:0/3960027140 wait complete. 2024-02-21T23:54:05.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.383+0000 7f9483b1e640 1 Processor -- start 2024-02-21T23:54:05.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.383+0000 7f9483b1e640 1 -- start start 2024-02-21T23:54:05.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.384+0000 7f9483b1e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0712b0 0x7f947c156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:05.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.384+0000 7f9483b1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f947c0cdfe0 0x7f947c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:05.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.384+0000 7f9483b1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0ce3c0 0x7f947c1f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:05.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.384+0000 7f9483b1e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c158930 con 0x7f947c0cdfe0 2024-02-21T23:54:05.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.384+0000 7f9483b1e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c158650 con 0x7f947c0712b0 2024-02-21T23:54:05.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.384+0000 7f9483b1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c14ea50 con 0x7f947c0ce3c0 2024-02-21T23:54:05.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.384+0000 7f94810b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f947c0cdfe0 0x7f947c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:05.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.385+0000 7f94818ba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0712b0 0x7f947c156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:05.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.385+0000 7f94820bb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0ce3c0 0x7f947c1f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:05.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.385+0000 7f94810b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f947c0cdfe0 0x7f947c0ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45888/0 (socket says 172.21.15.133:45888) 2024-02-21T23:54:05.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.385+0000 7f94818ba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0712b0 0x7f947c156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:54364/0 (socket says 172.21.15.133:54364) 2024-02-21T23:54:05.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.385+0000 7f94810b9640 1 -- 172.21.15.133:0/3622152337 learned_addr learned my addr 172.21.15.133:0/3622152337 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:05.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.388+0000 7f94810b9640 1 -- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0ce3c0 msgr2=0x7f947c1f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:05.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.388+0000 7f94810b9640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0ce3c0 0x7f947c1f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.388+0000 7f94810b9640 1 -- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0712b0 msgr2=0x7f947c156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:05.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.388+0000 7f94810b9640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0712b0 0x7f947c156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:05.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.388+0000 7f94810b9640 1 -- 172.21.15.133:0/3622152337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c06de40 con 0x7f947c0cdfe0 2024-02-21T23:54:05.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.388+0000 7f94818ba640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0712b0 0x7f947c156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-21T23:54:05.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.390+0000 7f94810b9640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f947c0cdfe0 0x7f947c0ccc20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f946c00e870 tx=0x7f946c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:05.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.394+0000 7f94727fc640 1 -- 172.21.15.133:0/3622152337 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c017070 con 0x7f947c0cdfe0 2024-02-21T23:54:05.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.394+0000 7f94727fc640 1 -- 172.21.15.133:0/3622152337 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f946c00c940 con 0x7f947c0cdfe0 2024-02-21T23:54:05.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.394+0000 7f94727fc640 1 -- 172.21.15.133:0/3622152337 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c00cd30 con 0x7f947c0cdfe0 2024-02-21T23:54:05.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.394+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c06db80 con 0x7f947c0cdfe0 2024-02-21T23:54:05.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.394+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c0b01b0 con 0x7f947c0cdfe0 2024-02-21T23:54:05.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.395+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c06a130 con 0x7f947c0cdfe0 2024-02-21T23:54:05.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.640+0000 7f94727fc640 1 -- 172.21.15.133:0/3622152337 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f946c01a070 con 0x7f947c0cdfe0 2024-02-21T23:54:05.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.640+0000 7f94727fc640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f946006b4d0 0x7f946006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:05.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.641+0000 7f94818ba640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f946006b4d0 0x7f946006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:05.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.641+0000 7f94818ba640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f946006b4d0 0x7f946006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9478035000 tx=0x7f9478030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:05.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.654+0000 7f94727fc640 1 -- 172.21.15.133:0/3622152337 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6908+0+0 (secure 0 0 0) 0x7f946c057120 con 0x7f947c0cdfe0 2024-02-21T23:54:05.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.654+0000 7f94727fc640 1 -- 172.21.15.133:0/3622152337 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f946c059080 con 0x7f947c0cdfe0 2024-02-21T23:54:05.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:05.863+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f947c152400 con 0x7f947c0cdfe0 2024-02-21T23:54:06.404 INFO:tasks.ceph.mds.f.smithi142.stderr:==00:00:29:03.036 41176== 2024-02-21T23:54:06.404 INFO:tasks.ceph.mds.f.smithi142.stderr:==00:00:29:03.036 41176== Exit program on first error (--exit-on-first-error=yes) 2024-02-21T23:54:06.437 INFO:tasks.ceph.mds.f.smithi142.stderr:daemon-helper: command failed with exit status 42 2024-02-21T23:54:06.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:06.998+0000 7f94727fc640 1 -- 172.21.15.133:0/3622152337 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f946c05d120 con 0x7f947c0cdfe0 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f946006b4d0 msgr2=0x7f946006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f946006b4d0 0x7f946006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9478035000 tx=0x7f9478030000 comp rx=0 tx=0).stop 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f947c0cdfe0 msgr2=0x7f947c0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f947c0cdfe0 0x7f947c0ccc20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f946c00e870 tx=0x7f946c00ed30 comp rx=0 tx=0).stop 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 shutdown_connections 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f946006b4d0 0x7f946006d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f947c0ce3c0 0x7f947c1f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f947c0cdfe0 0x7f947c0ccc20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 --2- 172.21.15.133:0/3622152337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f947c0712b0 0x7f947c156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 >> 172.21.15.133:0/3622152337 conn(0x7f947c06aec0 msgr2=0x7f947c074140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:06.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 shutdown_connections 2024-02-21T23:54:06.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.000+0000 7f9483b1e640 1 -- 172.21.15.133:0/3622152337 wait complete. 2024-02-21T23:54:07.006 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-21T23:54:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.119+0000 7fc98fa95640 1 Processor -- start 2024-02-21T23:54:07.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.119+0000 7fc98fa95640 1 -- start start 2024-02-21T23:54:07.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.120+0000 7fc98fa95640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc988154120 0x7fc9880c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:07.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.120+0000 7fc98fa95640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9880c4600 0x7fc98815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:07.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.120+0000 7fc98fa95640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9880c3760 con 0x7fc98815ac10 2024-02-21T23:54:07.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.120+0000 7fc98fa95640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9880c3a00 con 0x7fc988154120 2024-02-21T23:54:07.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.120+0000 7fc98fa95640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9880c3ce0 con 0x7fc9880c4600 2024-02-21T23:54:07.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.121+0000 7fc98d831640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc988154120 0x7fc9880c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:07.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.121+0000 7fc98e032640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fc98815ac10 0x7fc9880cd900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.133:56458/0 (socket says 172.21.15.133:56458) 2024-02-21T23:54:07.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.121+0000 7fc98e032640 1 -- 172.21.15.133:0/2496769748 learned_addr learned my addr 172.21.15.133:0/2496769748 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-02-21T23:54:07.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.121+0000 7fc98d030640 1 --2- 172.21.15.133:0/2496769748 >> v2:172.21.15.142:3300/0 conn(0x7fc9880c4600 0x7fc98815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:07.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.123+0000 7fc98d831640 1 -- 172.21.15.133:0/2496769748 >> v2:172.21.15.142:3300/0 conn(0x7fc9880c4600 msgr2=0x7fc98815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:07.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.123+0000 7fc98d831640 1 --2- 172.21.15.133:0/2496769748 >> v2:172.21.15.142:3300/0 conn(0x7fc9880c4600 0x7fc98815a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:07.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.123+0000 7fc98d831640 1 -- 172.21.15.133:0/2496769748 >> v1:172.21.15.142:6789/0 conn(0x7fc98815ac10 legacy=0x7fc9880cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:07.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.123+0000 7fc98d831640 1 -- 172.21.15.133:0/2496769748 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880c3fc0 con 0x7fc988154120 2024-02-21T23:54:07.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.125+0000 7fc98d831640 1 --2- 172.21.15.133:0/2496769748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc988154120 0x7fc9880c8900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc98806c2f0 tx=0x7fc98402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8814a4dce6571603 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:07.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.129+0000 7fc98c82f640 1 -- 172.21.15.133:0/2496769748 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984034070 con 0x7fc988154120 2024-02-21T23:54:07.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.129+0000 7fc98c82f640 1 -- 172.21.15.133:0/2496769748 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc984030610 con 0x7fc988154120 2024-02-21T23:54:07.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.129+0000 7fc98c82f640 1 -- 172.21.15.133:0/2496769748 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984030a00 con 0x7fc988154120 2024-02-21T23:54:07.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.129+0000 7fc98fa95640 1 -- 172.21.15.133:0/2496769748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc988154120 msgr2=0x7fc9880c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:07.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.129+0000 7fc98fa95640 1 --2- 172.21.15.133:0/2496769748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc988154120 0x7fc9880c8900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc98806c2f0 tx=0x7fc98402b960 comp rx=0 tx=0).stop 2024-02-21T23:54:07.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.129+0000 7fc98fa95640 1 -- 172.21.15.133:0/2496769748 shutdown_connections 2024-02-21T23:54:07.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.129+0000 7fc98fa95640 1 --2- 172.21.15.133:0/2496769748 >> v2:172.21.15.142:3300/0 conn(0x7fc9880c4600 0x7fc98815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:07.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.129+0000 7fc98fa95640 1 --2- 172.21.15.133:0/2496769748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc988154120 0x7fc9880c8900 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:07.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.129+0000 7fc98fa95640 1 -- 172.21.15.133:0/2496769748 >> 172.21.15.133:0/2496769748 conn(0x7fc98806f7f0 msgr2=0x7fc988061970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:07.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.130+0000 7fc98fa95640 1 -- 172.21.15.133:0/2496769748 shutdown_connections 2024-02-21T23:54:07.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.130+0000 7fc98fa95640 1 -- 172.21.15.133:0/2496769748 wait complete. 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.131+0000 7fc98fa95640 1 Processor -- start 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.131+0000 7fc98fa95640 1 -- start start 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.131+0000 7fc98fa95640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9880c4600 0x7fc98815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.131+0000 7fc98fa95640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc988154120 0x7fc988159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.132+0000 7fc98fa95640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc98815ac10 0x7fc988159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.132+0000 7fc98fa95640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880c4bd0 con 0x7fc988154120 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.132+0000 7fc98fa95640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880c4a50 con 0x7fc9880c4600 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.132+0000 7fc98fa95640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880ce010 con 0x7fc98815ac10 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.132+0000 7fc98d831640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9880c4600 0x7fc98815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.132+0000 7fc98d030640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc988154120 0x7fc988159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.132+0000 7fc98e032640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc98815ac10 0x7fc988159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:07.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.132+0000 7fc98d831640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9880c4600 0x7fc98815d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:54380/0 (socket says 172.21.15.133:54380) 2024-02-21T23:54:07.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.132+0000 7fc98d831640 1 -- 172.21.15.133:0/3375762950 learned_addr learned my addr 172.21.15.133:0/3375762950 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:07.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.135+0000 7fc98d831640 1 -- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc98815ac10 msgr2=0x7fc988159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:07.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.135+0000 7fc98d831640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc98815ac10 0x7fc988159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:07.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.135+0000 7fc98d831640 1 -- 172.21.15.133:0/3375762950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc988154120 msgr2=0x7fc988159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:07.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.135+0000 7fc98d831640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc988154120 0x7fc988159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:07.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.135+0000 7fc98d831640 1 -- 172.21.15.133:0/3375762950 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9881498b0 con 0x7fc9880c4600 2024-02-21T23:54:07.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.138+0000 7fc98d831640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9880c4600 0x7fc98815d9e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc984000c00 tx=0x7fc9840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:07.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.141+0000 7fc97e7fc640 1 -- 172.21.15.133:0/3375762950 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984034040 con 0x7fc9880c4600 2024-02-21T23:54:07.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.141+0000 7fc97e7fc640 1 -- 172.21.15.133:0/3375762950 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc984030410 con 0x7fc9880c4600 2024-02-21T23:54:07.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.141+0000 7fc97e7fc640 1 -- 172.21.15.133:0/3375762950 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984039c70 con 0x7fc9880c4600 2024-02-21T23:54:07.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.141+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9880c78c0 con 0x7fc9880c4600 2024-02-21T23:54:07.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.142+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc988149fc0 con 0x7fc9880c4600 2024-02-21T23:54:07.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.142+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98806a130 con 0x7fc9880c4600 2024-02-21T23:54:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.384+0000 7fc97e7fc640 1 -- 172.21.15.133:0/3375762950 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc984037070 con 0x7fc9880c4600 2024-02-21T23:54:07.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.385+0000 7fc97e7fc640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc96c06b4d0 0x7fc96c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:07.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.385+0000 7fc98d030640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc96c06b4d0 0x7fc96c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:07.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.385+0000 7fc98d030640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc96c06b4d0 0x7fc96c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc9880c41b0 tx=0x7fc978009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:07.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.389+0000 7fc97e7fc640 1 -- 172.21.15.133:0/3375762950 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6908+0+0 (secure 0 0 0) 0x7fc9840305c0 con 0x7fc9880c4600 2024-02-21T23:54:07.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.400+0000 7fc97e7fc640 1 -- 172.21.15.133:0/3375762950 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7fc9881498b0 con 0x7fc9880c4600 2024-02-21T23:54:07.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:07.571+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc9880b3b20 con 0x7fc9880c4600 2024-02-21T23:54:08.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2024-02-21T23:54:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.164+0000 7fc97e7fc640 1 -- 172.21.15.133:0/3375762950 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc98407b470 con 0x7fc9880c4600 2024-02-21T23:54:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-02-21T23:54:08.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.166+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc96c06b4d0 msgr2=0x7fc96c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.166+0000 7fc98fa95640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc96c06b4d0 0x7fc96c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc9880c41b0 tx=0x7fc978009190 comp rx=0 tx=0).stop 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.166+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9880c4600 msgr2=0x7fc98815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.166+0000 7fc98fa95640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9880c4600 0x7fc98815d9e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc984000c00 tx=0x7fc9840046c0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.167+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 shutdown_connections 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.167+0000 7fc98fa95640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7fc96c06b4d0 0x7fc96c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.167+0000 7fc98fa95640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc98815ac10 0x7fc988159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.167+0000 7fc98fa95640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc988154120 0x7fc988159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.167+0000 7fc98fa95640 1 --2- 172.21.15.133:0/3375762950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9880c4600 0x7fc98815d9e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.167+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 >> 172.21.15.133:0/3375762950 conn(0x7fc98806f7f0 msgr2=0x7fc988152530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.167+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 shutdown_connections 2024-02-21T23:54:08.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.167+0000 7fc98fa95640 1 -- 172.21.15.133:0/3375762950 wait complete. 2024-02-21T23:54:08.174 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-21T23:54:08.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.283+0000 7f08b798c640 1 Processor -- start 2024-02-21T23:54:08.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.283+0000 7f08b798c640 1 -- start start 2024-02-21T23:54:08.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.284+0000 7f08b798c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f08b00b0fe0 0x7f08b0072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:08.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.284+0000 7f08b798c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0072f80 0x7f08b006de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:08.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.284+0000 7f08b798c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b015afd0 con 0x7f08b0154300 2024-02-21T23:54:08.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.284+0000 7f08b798c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b015a170 con 0x7f08b00b0fe0 2024-02-21T23:54:08.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.284+0000 7f08b798c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b015a450 con 0x7f08b0072f80 2024-02-21T23:54:08.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.284+0000 7f08b5f29640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0072f80 0x7f08b006de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:08.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.285+0000 7f08b5f29640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0072f80 0x7f08b006de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:41318/0 (socket says 172.21.15.133:41318) 2024-02-21T23:54:08.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.285+0000 7f08b5f29640 1 -- 172.21.15.133:0/1564471523 learned_addr learned my addr 172.21.15.133:0/1564471523 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:08.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.285+0000 7f08b4f27640 1 --2- 172.21.15.133:0/1564471523 >> v2:172.21.15.142:3300/0 conn(0x7f08b00b0fe0 0x7f08b0072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.287+0000 7f08b5f29640 1 -- 172.21.15.133:0/1564471523 >> v2:172.21.15.142:3300/0 conn(0x7f08b00b0fe0 msgr2=0x7f08b0072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.287+0000 7f08b5f29640 1 --2- 172.21.15.133:0/1564471523 >> v2:172.21.15.142:3300/0 conn(0x7f08b00b0fe0 0x7f08b0072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.287+0000 7f08b5f29640 1 -- 172.21.15.133:0/1564471523 >> v1:172.21.15.142:6789/0 conn(0x7f08b0154300 legacy=0x7f08b006d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.287+0000 7f08b5f29640 1 -- 172.21.15.133:0/1564471523 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b015a730 con 0x7f08b0072f80 2024-02-21T23:54:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.289+0000 7f08b5f29640 1 --2- 172.21.15.133:0/1564471523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0072f80 0x7f08b006de00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009e50 tx=0x7f08a802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c46048ab89081dff server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.293+0000 7f08a7fff640 1 -- 172.21.15.133:0/1564471523 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a802cb90 con 0x7f08b0072f80 2024-02-21T23:54:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.293+0000 7f08a7fff640 1 -- 172.21.15.133:0/1564471523 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f08a802cd30 con 0x7f08b0072f80 2024-02-21T23:54:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.293+0000 7f08a7fff640 1 -- 172.21.15.133:0/1564471523 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8034630 con 0x7f08b0072f80 2024-02-21T23:54:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.294+0000 7f08b798c640 1 -- 172.21.15.133:0/1564471523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0072f80 msgr2=0x7f08b006de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.294+0000 7f08b798c640 1 --2- 172.21.15.133:0/1564471523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0072f80 0x7f08b006de00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009e50 tx=0x7f08a802bff0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.294+0000 7f08b798c640 1 -- 172.21.15.133:0/1564471523 shutdown_connections 2024-02-21T23:54:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.294+0000 7f08b798c640 1 --2- 172.21.15.133:0/1564471523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0072f80 0x7f08b006de00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.294+0000 7f08b798c640 1 --2- 172.21.15.133:0/1564471523 >> v2:172.21.15.142:3300/0 conn(0x7f08b00b0fe0 0x7f08b0072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.294+0000 7f08b798c640 1 -- 172.21.15.133:0/1564471523 >> 172.21.15.133:0/1564471523 conn(0x7f08b00cc620 msgr2=0x7f08b00c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.294+0000 7f08b798c640 1 -- 172.21.15.133:0/1564471523 shutdown_connections 2024-02-21T23:54:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.294+0000 7f08b798c640 1 -- 172.21.15.133:0/1564471523 wait complete. 2024-02-21T23:54:08.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.295+0000 7f08b798c640 1 Processor -- start 2024-02-21T23:54:08.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.295+0000 7f08b798c640 1 -- start start 2024-02-21T23:54:08.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.296+0000 7f08b798c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08b0072f80 0x7f08b00cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:08.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.296+0000 7f08b798c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b00b0fe0 0x7f08b01f3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:08.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.296+0000 7f08b798c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0154300 0x7f08b01599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:08.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.296+0000 7f08b798c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b01546e0 con 0x7f08b00b0fe0 2024-02-21T23:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.296+0000 7f08b798c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b015acf0 con 0x7f08b0072f80 2024-02-21T23:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.296+0000 7f08b798c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00c7640 con 0x7f08b0154300 2024-02-21T23:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.297+0000 7f08b4f27640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b00b0fe0 0x7f08b01f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.297+0000 7f08b5728640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08b0072f80 0x7f08b00cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.297+0000 7f08b5f29640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0154300 0x7f08b01599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.297+0000 7f08b5728640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08b0072f80 0x7f08b00cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:54392/0 (socket says 172.21.15.133:54392) 2024-02-21T23:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.297+0000 7f08b5728640 1 -- 172.21.15.133:0/2006938081 learned_addr learned my addr 172.21.15.133:0/2006938081 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.300+0000 7f08b5728640 1 -- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0154300 msgr2=0x7f08b01599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.300+0000 7f08b5728640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0154300 0x7f08b01599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.300+0000 7f08b5728640 1 -- 172.21.15.133:0/2006938081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b00b0fe0 msgr2=0x7f08b01f3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.300+0000 7f08b5728640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b00b0fe0 0x7f08b01f3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.300+0000 7f08b5728640 1 -- 172.21.15.133:0/2006938081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b0151c50 con 0x7f08b0072f80 2024-02-21T23:54:08.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.302+0000 7f08b5728640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08b0072f80 0x7f08b00cba80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00c3c0 tx=0x7f08ac00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.306+0000 7f08a5ffb640 1 -- 172.21.15.133:0/2006938081 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0052d0 con 0x7f08b0072f80 2024-02-21T23:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.306+0000 7f08a5ffb640 1 -- 172.21.15.133:0/2006938081 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f08ac005490 con 0x7f08b0072f80 2024-02-21T23:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.306+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b0151f10 con 0x7f08b0072f80 2024-02-21T23:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.306+0000 7f08a5ffb640 1 -- 172.21.15.133:0/2006938081 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac007280 con 0x7f08b0072f80 2024-02-21T23:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.306+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b00cafa0 con 0x7f08b0072f80 2024-02-21T23:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.307+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b006a130 con 0x7f08b0072f80 2024-02-21T23:54:08.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.326+0000 7f08a5ffb640 1 -- 172.21.15.133:0/2006938081 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08ac007420 con 0x7f08b0072f80 2024-02-21T23:54:08.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.326+0000 7f08a5ffb640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f089406b4d0 0x7f089406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:08.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.326+0000 7f08b4f27640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f089406b4d0 0x7f089406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:08.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.327+0000 7f08b4f27640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f089406b4d0 0x7f089406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f08b006e600 tx=0x7f08a0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:08.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.328+0000 7f08a5ffb640 1 -- 172.21.15.133:0/2006938081 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6470+0+0 (secure 0 0 0) 0x7f08ac056450 con 0x7f08b0072f80 2024-02-21T23:54:08.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.565+0000 7f08a5ffb640 1 -- 172.21.15.133:0/2006938081 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f08ac0583b0 con 0x7f08b0072f80 2024-02-21T23:54:08.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:08.742+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f08b006c880 con 0x7f08b0072f80 2024-02-21T23:54:09.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.158+0000 7f08a5ffb640 1 -- 172.21.15.133:0/2006938081 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f08ac05c450 con 0x7f08b0072f80 2024-02-21T23:54:09.152 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-02-21T23:54:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f089406b4d0 msgr2=0x7f089406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f089406b4d0 0x7f089406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f08b006e600 tx=0x7f08a0009190 comp rx=0 tx=0).stop 2024-02-21T23:54:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08b0072f80 msgr2=0x7f08b00cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08b0072f80 0x7f08b00cba80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00c3c0 tx=0x7f08ac00c880 comp rx=0 tx=0).stop 2024-02-21T23:54:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 shutdown_connections 2024-02-21T23:54:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f089406b4d0 0x7f089406d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08b0154300 0x7f08b01599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b00b0fe0 0x7f08b01f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 --2- 172.21.15.133:0/2006938081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08b0072f80 0x7f08b00cba80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 >> 172.21.15.133:0/2006938081 conn(0x7f08b00cc620 msgr2=0x7f08b014fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 shutdown_connections 2024-02-21T23:54:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.160+0000 7f08b798c640 1 -- 172.21.15.133:0/2006938081 wait complete. 2024-02-21T23:54:09.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-02-21T23:54:09.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.287+0000 7f9d0b4a6640 1 Processor -- start 2024-02-21T23:54:09.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.287+0000 7f9d0b4a6640 1 -- start start 2024-02-21T23:54:09.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.288+0000 7f9d0b4a6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d040ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:09.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.288+0000 7f9d0b4a6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d04070800 0x7f9d0415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:09.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.288+0000 7f9d0b4a6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040cd300 con 0x7f9d040703c0 2024-02-21T23:54:09.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.288+0000 7f9d0b4a6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d040cd570 con 0x7f9d04070800 2024-02-21T23:54:09.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.288+0000 7f9d0b4a6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0415a130 con 0x7f9d04154240 2024-02-21T23:54:09.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.288+0000 7f9d09242640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d040ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:09.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.289+0000 7f9d09242640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d040ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.133:41344/0 (socket says 172.21.15.133:41344) 2024-02-21T23:54:09.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.289+0000 7f9d09242640 1 -- 172.21.15.133:0/2653976948 learned_addr learned my addr 172.21.15.133:0/2653976948 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:09.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.291+0000 7f9d09242640 1 -- 172.21.15.133:0/2653976948 >> v2:172.21.15.133:3300/0 conn(0x7f9d04070800 msgr2=0x7f9d0415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:09.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.291+0000 7f9d09242640 1 --2- 172.21.15.133:0/2653976948 >> v2:172.21.15.133:3300/0 conn(0x7f9d04070800 0x7f9d0415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.291+0000 7f9d09242640 1 -- 172.21.15.133:0/2653976948 >> v1:172.21.15.133:6789/0 conn(0x7f9d040703c0 legacy=0x7f9d040c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-21T23:54:09.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.291+0000 7f9d09242640 1 -- 172.21.15.133:0/2653976948 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0415a410 con 0x7f9d04154240 2024-02-21T23:54:09.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.293+0000 7f9d09242640 1 --2- 172.21.15.133:0/2653976948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d040ccdc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009870 tx=0x7f9d0002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e2cd93d6c544a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:09.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.297+0000 7f9cfbfff640 1 -- 172.21.15.133:0/2653976948 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00035020 con 0x7f9d04154240 2024-02-21T23:54:09.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.297+0000 7f9cfbfff640 1 -- 172.21.15.133:0/2653976948 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9d00031640 con 0x7f9d04154240 2024-02-21T23:54:09.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.297+0000 7f9cfbfff640 1 -- 172.21.15.133:0/2653976948 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0002d040 con 0x7f9d04154240 2024-02-21T23:54:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.298+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/2653976948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 msgr2=0x7f9d040ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.298+0000 7f9d0b4a6640 1 --2- 172.21.15.133:0/2653976948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d040ccdc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009870 tx=0x7f9d0002b8f0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.298+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/2653976948 shutdown_connections 2024-02-21T23:54:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.298+0000 7f9d0b4a6640 1 --2- 172.21.15.133:0/2653976948 >> v2:172.21.15.133:3300/0 conn(0x7f9d04070800 0x7f9d0415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.298+0000 7f9d0b4a6640 1 --2- 172.21.15.133:0/2653976948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d040ccdc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.298+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/2653976948 >> 172.21.15.133:0/2653976948 conn(0x7f9d04061b50 msgr2=0x7f9d04072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.298+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/2653976948 shutdown_connections 2024-02-21T23:54:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.298+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/2653976948 wait complete. 2024-02-21T23:54:09.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.299+0000 7f9d0b4a6640 1 Processor -- start 2024-02-21T23:54:09.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.299+0000 7f9d0b4a6640 1 -- start start 2024-02-21T23:54:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.300+0000 7f9d0b4a6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d040703c0 0x7f9d040cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.300+0000 7f9d0b4a6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d04070800 0x7f9d040cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.300+0000 7f9d0b4a6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d041f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.300+0000 7f9d0b4a6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04154620 con 0x7f9d040703c0 2024-02-21T23:54:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.300+0000 7f9d0b4a6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0415a9d0 con 0x7f9d04070800 2024-02-21T23:54:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.300+0000 7f9d0b4a6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040cac90 con 0x7f9d04154240 2024-02-21T23:54:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.300+0000 7f9d08a41640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d04070800 0x7f9d040cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.301+0000 7f9d09a43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d041f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.301+0000 7f9d08a41640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d04070800 0x7f9d040cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.133:54394/0 (socket says 172.21.15.133:54394) 2024-02-21T23:54:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.301+0000 7f9d08a41640 1 -- 172.21.15.133:0/1269224167 learned_addr learned my addr 172.21.15.133:0/1269224167 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-02-21T23:54:09.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.304+0000 7f9d08a41640 1 -- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 msgr2=0x7f9d041f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:09.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.304+0000 7f9d08a41640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d041f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.304+0000 7f9d08a41640 1 -- 172.21.15.133:0/1269224167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d040703c0 msgr2=0x7f9d040cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:09.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.304+0000 7f9d08a41640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d040703c0 0x7f9d040cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:09.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.304+0000 7f9d08a41640 1 -- 172.21.15.133:0/1269224167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0414f710 con 0x7f9d04070800 2024-02-21T23:54:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.306+0000 7f9d08a41640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d04070800 0x7f9d040cb660 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b3e0 tx=0x7f9cf400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.310+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/1269224167 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4015020 con 0x7f9d04070800 2024-02-21T23:54:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.310+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/1269224167 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9cf4011850 con 0x7f9d04070800 2024-02-21T23:54:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.310+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/1269224167 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4011c60 con 0x7f9d04070800 2024-02-21T23:54:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.310+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0414f9f0 con 0x7f9d04070800 2024-02-21T23:54:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.311+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d040ca920 con 0x7f9d04070800 2024-02-21T23:54:09.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.312+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0414ff10 con 0x7f9d04070800 2024-02-21T23:54:09.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.329+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/1269224167 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9cf4018020 con 0x7f9d04070800 2024-02-21T23:54:09.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.330+0000 7f9cf9ffb640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9ce806b550 0x7f9ce806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-21T23:54:09.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.330+0000 7f9d09242640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9ce806b550 0x7f9ce806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-21T23:54:09.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.331+0000 7f9d09242640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9ce806b550 0x7f9ce806da10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d00036000 tx=0x7f9d00030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-21T23:54:09.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.333+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/1269224167 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f9cf4011e00 con 0x7f9d04070800 2024-02-21T23:54:09.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.574+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/1269224167 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183725 (secure 0 0 0) 0x7f9d0414ff10 con 0x7f9d04070800 2024-02-21T23:54:09.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:09.747+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9d0406a130 con 0x7f9d04070800 2024-02-21T23:54:10.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.388+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/1269224167 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f9cf401f070 con 0x7f9d04070800 2024-02-21T23:54:10.383 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data_ec' removed 2024-02-21T23:54:10.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9ce806b550 msgr2=0x7f9ce806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:10.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9ce806b550 0x7f9ce806da10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d00036000 tx=0x7f9d00030000 comp rx=0 tx=0).stop 2024-02-21T23:54:10.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d04070800 msgr2=0x7f9d040cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-21T23:54:10.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d04070800 0x7f9d040cb660 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b3e0 tx=0x7f9cf400b8a0 comp rx=0 tx=0).stop 2024-02-21T23:54:10.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 shutdown_connections 2024-02-21T23:54:10.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:6800/3089949271,v1:172.21.15.142:6801/3089949271] conn(0x7f9ce806b550 0x7f9ce806da10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:10.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d04154240 0x7f9d041f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:10.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d04070800 0x7f9d040cb660 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:10.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 --2- 172.21.15.133:0/1269224167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d040703c0 0x7f9d040cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-21T23:54:10.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.391+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 >> 172.21.15.133:0/1269224167 conn(0x7f9d04061b50 msgr2=0x7f9d04158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-21T23:54:10.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.392+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 shutdown_connections 2024-02-21T23:54:10.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-02-21T23:54:10.392+0000 7f9d0b4a6640 1 -- 172.21.15.133:0/1269224167 wait complete. 2024-02-21T23:54:10.400 INFO:teuthology.misc:Shutting down mds daemons... 2024-02-21T23:54:10.400 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-21T23:54:10.400 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:10.412 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-02-21T23:54:10.410+0000 b7e4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i a (PID: 41130) UID: 0 2024-02-21T23:54:10.413 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-02-21T23:54:10.411+0000 b7e4640 -1 mds.a *** got signal Terminated *** 2024-02-21T23:54:14.822 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~7s 2024-02-21T23:54:16.503 INFO:tasks.ceph.mds.a:Stopped 2024-02-21T23:54:16.504 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-21T23:54:16.504 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:16.518 INFO:tasks.ceph.mds.c.smithi133.stderr:2024-02-21T23:54:16.513+0000 b7e4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i c (PID: 41135) UID: 0 2024-02-21T23:54:16.519 INFO:tasks.ceph.mds.c.smithi133.stderr:2024-02-21T23:54:16.515+0000 b7e4640 -1 mds.c *** got signal Terminated *** 2024-02-21T23:54:21.436 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-02-21T23:54:21.436 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~14s 2024-02-21T23:54:22.605 INFO:tasks.ceph.mds.c:Stopped 2024-02-21T23:54:22.606 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-02-21T23:54:22.606 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:22.620 INFO:tasks.ceph.mds.e.smithi133.stderr:2024-02-21T23:54:22.614+0000 b7e4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.e.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i e (PID: 41141) UID: 0 2024-02-21T23:54:22.621 INFO:tasks.ceph.mds.e.smithi133.stderr:2024-02-21T23:54:22.616+0000 b7e4640 -1 mds.e *** got signal Terminated *** 2024-02-21T23:54:27.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2024-02-21T23:54:27.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~20s 2024-02-21T23:54:27.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-02-21T23:54:28.707 INFO:tasks.ceph.mds.e:Stopped 2024-02-21T23:54:28.708 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-21T23:54:28.708 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:28.723 INFO:tasks.ceph.mds.b.smithi142.stderr:2024-02-21T23:54:28.717+0000 b7e4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i b (PID: 41167) UID: 0 2024-02-21T23:54:28.724 INFO:tasks.ceph.mds.b.smithi142.stderr:2024-02-21T23:54:28.719+0000 b7e4640 -1 mds.b *** got signal Terminated *** 2024-02-21T23:54:33.690 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:54:33.692 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:54:34.361 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-02-21T23:54:34.361 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~27s 2024-02-21T23:54:34.361 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e has been restored 2024-02-21T23:54:34.810 INFO:tasks.ceph.mds.b:Stopped 2024-02-21T23:54:34.810 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-21T23:54:34.811 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:34.825 INFO:tasks.ceph.mds.d.smithi142.stderr:2024-02-21T23:54:34.819+0000 b7e4640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.d.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i d (PID: 41170) UID: 0 2024-02-21T23:54:34.826 INFO:tasks.ceph.mds.d.smithi142.stderr:2024-02-21T23:54:34.821+0000 b7e4640 -1 mds.d *** got signal Terminated *** 2024-02-21T23:54:40.670 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-02-21T23:54:40.670 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~33s 2024-02-21T23:54:40.670 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-02-21T23:54:40.913 INFO:tasks.ceph.mds.d:Stopped 2024-02-21T23:54:40.913 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-02-21T23:54:40.913 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:40.913 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-02-21T23:54:40.914 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi142 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.f.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i f" 2024-02-21T23:54:40.914 INFO:tasks.ceph.mds.f:Stopped 2024-02-21T23:54:40.914 INFO:teuthology.misc:Shutting down osd daemons... 2024-02-21T23:54:40.915 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-02-21T23:54:40.915 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:40.916 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-02-21T23:54:40.920+0000 7f441cbf9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 0 (PID: 37307) UID: 0 2024-02-21T23:54:40.916 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-02-21T23:54:40.920+0000 7f441cbf9640 -1 osd.0 35 *** Got signal Terminated *** 2024-02-21T23:54:46.880 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-02-21T23:54:46.881 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f has been restored 2024-02-21T23:54:46.881 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-02-21T23:54:47.017 INFO:tasks.ceph.osd.0:Stopped 2024-02-21T23:54:47.017 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-02-21T23:54:47.017 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:47.018 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-02-21T23:54:47.022+0000 7faecb7f9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 1 (PID: 37316) UID: 0 2024-02-21T23:54:47.018 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-02-21T23:54:47.022+0000 7faecb7f9640 -1 osd.1 37 *** Got signal Terminated *** 2024-02-21T23:54:52.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-02-21T23:54:52.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-02-21T23:54:53.118 INFO:tasks.ceph.osd.1:Stopped 2024-02-21T23:54:53.119 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-02-21T23:54:53.119 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:53.120 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-02-21T23:54:53.124+0000 7f29637f9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 2 (PID: 37318) UID: 0 2024-02-21T23:54:53.120 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-02-21T23:54:53.124+0000 7f29637f9640 -1 osd.2 38 *** Got signal Terminated *** 2024-02-21T23:54:58.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-02-21T23:54:58.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-02-21T23:54:59.221 INFO:tasks.ceph.osd.2:Stopped 2024-02-21T23:54:59.221 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-02-21T23:54:59.221 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:54:59.222 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-02-21T23:54:59.227+0000 7ff2477f9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 3 (PID: 37322) UID: 0 2024-02-21T23:54:59.222 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-02-21T23:54:59.227+0000 7ff2477f9640 -1 osd.3 39 *** Got signal Terminated *** 2024-02-21T23:55:03.804 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:55:03.807 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:55:04.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-02-21T23:55:04.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-02-21T23:55:05.323 INFO:tasks.ceph.osd.3:Stopped 2024-02-21T23:55:05.323 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-02-21T23:55:05.323 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:55:05.324 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-02-21T23:55:05.328+0000 7f54283f9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 4 (PID: 37509) UID: 0 2024-02-21T23:55:05.324 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-02-21T23:55:05.328+0000 7f54283f9640 -1 osd.4 40 *** Got signal Terminated *** 2024-02-21T23:55:10.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-02-21T23:55:10.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-02-21T23:55:11.425 INFO:tasks.ceph.osd.4:Stopped 2024-02-21T23:55:11.425 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-02-21T23:55:11.426 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:55:11.426 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-02-21T23:55:11.430+0000 7f6a231f9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 5 (PID: 37512) UID: 0 2024-02-21T23:55:11.426 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-02-21T23:55:11.430+0000 7f6a231f9640 -1 osd.5 41 *** Got signal Terminated *** 2024-02-21T23:55:16.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-02-21T23:55:16.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-02-21T23:55:17.527 INFO:tasks.ceph.osd.5:Stopped 2024-02-21T23:55:17.528 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-02-21T23:55:17.528 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:55:17.529 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-02-21T23:55:17.532+0000 7fd7579f9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 6 (PID: 37516) UID: 0 2024-02-21T23:55:17.529 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-02-21T23:55:17.532+0000 7fd7579f9640 -1 osd.6 42 *** Got signal Terminated *** 2024-02-21T23:55:22.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-02-21T23:55:22.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-02-21T23:55:23.629 INFO:tasks.ceph.osd.6:Stopped 2024-02-21T23:55:23.629 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-02-21T23:55:23.630 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:55:23.630 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-02-21T23:55:23.634+0000 7f60153f9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 7 (PID: 37514) UID: 0 2024-02-21T23:55:23.630 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-02-21T23:55:23.634+0000 7f60153f9640 -1 osd.7 42 *** Got signal Terminated *** 2024-02-21T23:55:27.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-02-21T23:55:27.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-02-21T23:55:29.731 INFO:tasks.ceph.osd.7:Stopped 2024-02-21T23:55:29.731 INFO:teuthology.misc:Shutting down mgr daemons... 2024-02-21T23:55:29.731 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-02-21T23:55:29.731 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:55:29.757 INFO:tasks.ceph.mgr.y:Stopped 2024-02-21T23:55:29.757 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-02-21T23:55:29.757 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:55:29.758 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:55:29.761+0000 7f0e4aff7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 36933) UID: 0 2024-02-21T23:55:29.758 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-02-21T23:55:29.761+0000 7f0e4aff7640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-21T23:55:29.785 INFO:tasks.ceph.mgr.x:Stopped 2024-02-21T23:55:29.785 INFO:teuthology.misc:Shutting down mon daemons... 2024-02-21T23:55:29.785 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-02-21T23:55:29.785 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:55:29.817 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-02-21T23:55:29.804+0000 a18d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 37012) UID: 0 2024-02-21T23:55:29.817 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-02-21T23:55:29.805+0000 a18d640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-21T23:55:30.462 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:30:58.359 37032== 2024-02-21T23:55:30.462 INFO:tasks.ceph.mon.a.smithi133.stderr:==00:00:30:58.359 37032== Exit program on first error (--exit-on-first-error=yes) 2024-02-21T23:55:30.474 INFO:tasks.ceph.mon.a.smithi133.stderr:daemon-helper: command failed with exit status 42 2024-02-21T23:55:32.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-02-21T23:55:32.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-02-21T23:55:33.947 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:55:33.950 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:55:35.886 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-02-21T23:55:35.887 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi133 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2024-02-21T23:55:35.887 INFO:tasks.ceph.mon.a:Stopped 2024-02-21T23:55:35.887 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-02-21T23:55:35.888 INFO:teuthology.orchestra.run:waiting for 300 2024-02-21T23:55:35.903 INFO:tasks.ceph.mon.b.smithi142.stderr:2024-02-21T23:55:35.896+0000 a18d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 36928) UID: 0 2024-02-21T23:55:35.904 INFO:tasks.ceph.mon.b.smithi142.stderr:2024-02-21T23:55:35.897+0000 a18d640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-02-21T23:55:37.934 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-02-21T23:56:04.077 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:56:04.080 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:56:34.163 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:56:34.166 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:57:04.453 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:57:04.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:57:34.592 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:57:34.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:58:04.762 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:58:04.765 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:58:34.825 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:58:34.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:59:04.953 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:59:04.957 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:59:35.100 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-21T23:59:35.103 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-22T00:00:05.223 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-22T00:00:05.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-22T00:00:34.961 INFO:tasks.ceph:Checking cluster log for badness... 2024-02-22T00:00:34.961 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v MON_DOWN | head -n 1 2024-02-22T00:00:35.087 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi133.front.sepia.ceph.com 2024-02-22T00:00:35.087 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-02-22T00:00:35.149 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi133.front.sepia.ceph.com 2024-02-22T00:00:35.149 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-02-22T00:00:35.205 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi133.front.sepia.ceph.com 2024-02-22T00:00:35.206 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-02-22T00:00:35.269 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi133.front.sepia.ceph.com 2024-02-22T00:00:35.269 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-02-22T00:00:35.341 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi142.front.sepia.ceph.com 2024-02-22T00:00:35.341 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-02-22T00:00:35.400 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-22T00:00:35.403 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-22T00:00:35.499 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi142.front.sepia.ceph.com 2024-02-22T00:00:35.499 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-02-22T00:00:35.569 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi142.front.sepia.ceph.com 2024-02-22T00:00:35.569 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-02-22T00:00:35.636 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi142.front.sepia.ceph.com 2024-02-22T00:00:35.637 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-02-22T00:00:35.700 INFO:tasks.ceph:Archiving mon data... 2024-02-22T00:00:35.701 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/data/mon.a.tgz 2024-02-22T00:00:35.701 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-02-22T00:00:35.716 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.NgFIk3Fbf5 2024-02-22T00:00:35.717 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.NgFIk3Fbf5 2024-02-22T00:00:35.860 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.NgFIk3Fbf5 2024-02-22T00:00:35.944 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.NgFIk3Fbf5 is 324KB 2024-02-22T00:00:36.040 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.NgFIk3Fbf5 2024-02-22T00:00:36.055 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/data/mon.b.tgz 2024-02-22T00:00:36.055 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-02-22T00:00:36.069 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.V1HDs9W1QE 2024-02-22T00:00:36.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.V1HDs9W1QE 2024-02-22T00:00:36.222 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.V1HDs9W1QE 2024-02-22T00:00:36.306 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.V1HDs9W1QE is 350KB 2024-02-22T00:00:36.406 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.V1HDs9W1QE 2024-02-22T00:00:36.420 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/data/mon.c.tgz 2024-02-22T00:00:36.420 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-02-22T00:00:36.472 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.zxozvIvPe1 2024-02-22T00:00:36.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.zxozvIvPe1 2024-02-22T00:00:36.607 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.zxozvIvPe1 2024-02-22T00:00:36.691 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.zxozvIvPe1 is 351KB 2024-02-22T00:00:36.784 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.zxozvIvPe1 2024-02-22T00:00:36.798 INFO:tasks.ceph:Cleaning ceph cluster... 2024-02-22T00:00:36.798 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-02-22T00:00:36.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-02-22T00:00:37.135 DEBUG:tasks.ceph:valgrind exception message: valgrind error: Leak_StillReachable malloc malloc strdup 2024-02-22T00:00:37.347 ERROR:teuthology.util.scanner:Premature end of data in tag valgrindoutput line 3, line 52, column 1 (, line 52) 2024-02-22T00:00:37.392 ERROR:teuthology.util.scanner:Premature end of data in tag valgrindoutput line 3, line 52, column 1 (, line 52) 2024-02-22T00:00:37.394 INFO:tasks.ceph:Archiving crash dumps... 2024-02-22T00:00:37.396 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/remote/smithi133/crash 2024-02-22T00:00:37.397 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-02-22T00:00:37.424 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/remote/smithi142/crash 2024-02-22T00:00:37.425 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-02-22T00:00:37.450 INFO:tasks.ceph:Shutting down logrotate 2024-02-22T00:00:37.451 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-22T00:00:37.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-22T00:00:37.552 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-02-22T00:00:37.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-02-22T00:00:37.606 INFO:tasks.ceph:Compressing logs... 2024-02-22T00:00:37.606 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-22T00:00:37.622 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-22T00:00:37.719 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-02-22T00:00:37.719 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2024-02-22T00:00:37.719 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.f.log 2024-02-22T00:00:37.719 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/valgrind/mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.b.log 2024-02-22T00:00:37.720 INFO:teuthology.orchestra.run.smithi142.stderr: 55.9% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2024-02-22T00:00:37.720 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/valgrind/mds.f.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2024-02-22T00:00:37.720 INFO:teuthology.orchestra.run.smithi142.stderr: 67.8% -- replaced with /var/log/ceph/valgrind/mds.f.log.gz 2024-02-22T00:00:37.720 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/valgrind/mds.b.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.d.log 2024-02-22T00:00:37.720 INFO:teuthology.orchestra.run.smithi142.stderr: 57.2% -- replaced with /var/log/ceph/valgrind/mds.b.log.gz 2024-02-22T00:00:37.720 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose 55.7% -- -- replaced with /var/log/ceph/valgrind/mon.b.log.gz /var/log/ceph/ceph.audit.log 2024-02-22T00:00:37.721 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-02-22T00:00:37.721 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/valgrind/mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-02-22T00:00:37.721 INFO:teuthology.orchestra.run.smithi142.stderr: 57.4% -- replaced with /var/log/ceph/valgrind/mds.d.log.gz 2024-02-22T00:00:37.721 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-02-22T00:00:37.721 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37248.log 2024-02-22T00:00:37.721 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37368.log 2024-02-22T00:00:37.722 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37328.log 2024-02-22T00:00:37.722 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37368.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37248.log.gz 2024-02-22T00:00:37.722 INFO:teuthology.orchestra.run.smithi142.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-02-22T00:00:37.722 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-02-22T00:00:37.722 INFO:teuthology.orchestra.run.smithi142.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37368.log.gz 2024-02-22T00:00:37.723 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37328.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-02-22T00:00:37.723 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37328.log.gz 2024-02-22T00:00:37.723 INFO:teuthology.orchestra.run.smithi142.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37208.log 2024-02-22T00:00:37.723 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-02-22T00:00:37.723 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37419.log 2024-02-22T00:00:37.723 INFO:teuthology.orchestra.run.smithi142.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37208.log.gz 2024-02-22T00:00:37.723 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mds.f.log: 92.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-02-22T00:00:37.724 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37288.log 2024-02-22T00:00:37.724 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37419.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-02-22T00:00:37.724 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37288.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37419.log.gz 2024-02-22T00:00:37.724 INFO:teuthology.orchestra.run.smithi142.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37288.log.gz 2024-02-22T00:00:37.724 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37168.log 2024-02-22T00:00:37.724 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-02-22T00:00:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37168.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37168.log.gz 2024-02-22T00:00:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-02-22T00:00:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph.log: 93.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-02-22T00:00:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37128.log 2024-02-22T00:00:37.736 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-02-22T00:00:37.737 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37128.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37128.log.gz 2024-02-22T00:00:37.749 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-02-22T00:00:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43717.log 2024-02-22T00:00:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-02-22T00:00:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-02-22T00:00:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43717.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49349.log 2024-02-22T00:00:37.757 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43717.log.gz 2024-02-22T00:00:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41032.log 2024-02-22T00:00:37.757 INFO:teuthology.orchestra.run.smithi133.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45712.log 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49349.log: /var/log/ceph/ceph-client.admin.41032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49265.log 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49349.log.gz 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45712.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41032.log.gz 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42898.log 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44014.log 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45712.log.gz 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42898.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49265.log.gz 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44024.log 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44014.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42898.log.gz 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41789.log 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49307.log 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44014.log.gz 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41789.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44024.log.gz 2024-02-22T00:00:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43844.log 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41789.log.gz 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41621.log 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.43844.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44387.log 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49307.log.gz 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41621.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43844.log.gz 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48761.log 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4%/var/log/ceph/ceph-client.admin.44387.log: -- replaced with /var/log/ceph/ceph-client.admin.41621.log.gz 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.c.log 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3%/var/log/ceph/ceph-client.admin.48761.log: -- replaced with /var/log/ceph/ceph-client.admin.44387.log.gz 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.a.log 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/valgrind/mds.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/client.0.log 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48761.log.gz 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr: 57.2% -- replaced with /var/log/ceph/valgrind/mds.c.log.gz 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/valgrind/mds.a.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2024-02-22T00:00:37.759 INFO:teuthology.orchestra.run.smithi133.stderr: 57.3% -- replaced with /var/log/ceph/valgrind/mds.a.log.gz 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/valgrind/client.0.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.e.log 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr: 76.9% -- replaced with /var/log/ceph/valgrind/client.0.log.gz 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/valgrind/mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37051.log 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr: 69.6% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/valgrind/mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49560.log 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr: 57.3% -- replaced with /var/log/ceph/valgrind/mds.e.log.gz 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41831.log 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44954.log 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41831.log: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.37051.log.gz 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49560.log.gz 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49181.log 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41831.log.gz 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43292.log 2024-02-22T00:00:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44954.log: /var/log/ceph/ceph-client.admin.49181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45416.log 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44954.log.gz 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43292.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49181.log.gz 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41312.log 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45416.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43292.log.gz 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48563.log 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41312.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45416.log.gz 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41453.log 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48563.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41312.log.gz 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43668.log 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41453.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.44870.log -- replaced with /var/log/ceph/ceph-client.admin.48563.log.gz 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41453.log.gz 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41537.log 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44870.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43668.log.gz 2024-02-22T00:00:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44996.log 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44870.log.gz 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44021.log 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44996.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41537.log.gz 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43208.log 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44996.log.gz 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49055.log 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44025.log 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44021.log.gz 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43208.log.gz 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49476.log 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49055.log.gz 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49704.log 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49476.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44025.log.gz 2024-02-22T00:00:37.762 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42814.log 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49704.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49476.log.gz 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45500.log 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49704.log.gz 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45038.log 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45500.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42814.log.gz 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45500.log.gz 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49013.log 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41999.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45038.log.gz 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48648.log 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43370.log 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44912.log 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49013.log.gz 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48648.log.gz 2024-02-22T00:00:37.763 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45542.log 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43370.log.gz 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49139.log 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45542.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44912.log.gz 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41192.log 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49139.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45542.log.gz 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.48971.log 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49139.log.gz 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49223.log 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48971.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41192.log.gz 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48069.log 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49223.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48971.log.gz 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49830.log 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49223.log.gz 2024-02-22T00:00:37.764 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44210.log 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42982.log 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48069.log.gz 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49830.log.gz 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44210.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44210.log.gz 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44696.log 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42982.log.gz 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41663.log 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44696.log.gz 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43412.log 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41663.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41957.log 2024-02-22T00:00:37.765 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41663.log.gz 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48379.log 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43412.log.gz 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44022.log 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49391.log 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41957.log.gz 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48379.log.gz 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42856.log 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44022.log.gz 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48521.log 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44026.log 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49391.log.gz 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48521.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42856.log.gz 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45164.log 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48521.log.gz 2024-02-22T00:00:37.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41147.log 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41915.log 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44026.log.gz 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41147.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45164.log.gz 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41147.log.gz 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45080.log 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41915.log.gz 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44321.log 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45080.log.gz 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44777.log 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44321.log: /var/log/ceph/ceph-client.admin.48337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44676.log 2024-02-22T00:00:37.767 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44321.log.gz 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41747.log 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44676.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44777.log.gz 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37438.log 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44676.log.gz 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45248.log 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41747.log: /var/log/ceph/ceph-client.admin.37438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49097.log 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41747.log.gz 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45248.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37438.log.gz 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43622.log 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45248.log.gz 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44558.log 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49097.log: /var/log/ceph/ceph-client.admin.43622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48111.log 2024-02-22T00:00:37.768 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49097.log.gz 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44558.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43622.log.gz 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48253.log 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45206.log 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48111.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44558.log.gz 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48253.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48111.log.gz 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45332.log 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48253.log.gz 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44797.log 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45206.log: /var/log/ceph/ceph-client.admin.45332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44598.log 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45206.log.gz 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44797.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45332.log.gz 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44797.log.gz 2024-02-22T00:00:37.769 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47943.log 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44598.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47901.log 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47943.log: gzip 88.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44598.log.gz -- 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.42590.log 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47943.log.gz 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43496.log 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42101.log 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42590.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47901.log.gz 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48169.log 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43496.log.gz 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43454.log 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.48169.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41873.log 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45584.log 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr: 87.6% 90.7% -- replaced with /var/log/ceph/ceph-client.admin.42101.log.gz 2024-02-22T00:00:37.770 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48169.log.gz 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41873.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43454.log.gz 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43024.log 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45584.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.42590.log.gz 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41873.log.gz 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43024.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mds.e.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43024.log.gz 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45584.log.gz 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48295.log 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41579.log 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44023.log 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41579.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48295.log.gz 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41234.log 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41579.log.gz/var/log/ceph/ceph-client.admin.44023.log: 2024-02-22T00:00:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44296.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44023.log.gz 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43580.log 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41234.log.gz 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43538.log 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44296.log.gz/var/log/ceph/ceph-client.admin.43580.log: 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42167.log 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43580.log.gz 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45374.log 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43538.log.gz 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43250.log 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42167.log.gz 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44670.log 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45374.log: /var/log/ceph/ceph-client.admin.43250.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45374.log.gz 2024-02-22T00:00:37.772 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43250.log.gz 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41409.log 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44670.log.gz 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41409.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41409.log.gz 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44016.log: 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45458.log 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44242.log 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.0.log: gzip/var/log/ceph/ceph-client.admin.45458.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48211.log 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44242.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45458.log.gz 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46098.log 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44242.log.gz 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31035.log 2024-02-22T00:00:37.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48211.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48211.log.gz/var/log/ceph/ceph-client.0.46098.log: 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.tmp-client.admin.31035.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31035.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45122.log 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49436.log 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45122.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45122.log.gz 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48605.log 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49436.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49436.log.gz 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48605.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48605.log.gz 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47985.log 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.2.log: 88.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47985.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47985.log.gz 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48929.log 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mds.c.log: 92.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-02-22T00:00:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44320.log 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr: 94.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48929.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48929.log.gz 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48845.log 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43802.log 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48845.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44320.log.gz 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48845.log.gz 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42940.log 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43802.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43802.log.gz 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45290.log 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48887.log 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45290.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42940.log.gz 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45290.log.gz 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48027.log 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48887.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48887.log.gz 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49620.log 2024-02-22T00:00:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49788.log 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49620.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48027.log.gz 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49620.log.gz 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49746.log 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49788.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49788.log.gz 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43760.log 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49518.log 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43760.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49746.log.gz 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43760.log.gz 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48719.log 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49518.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49518.log.gz 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43082.log 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41495.log 2024-02-22T00:00:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43082.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48719.log.gz 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43082.log.gz 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44410.log 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41705.log 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44410.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41495.log.gz 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44410.log.gz 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42041.log 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41705.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41705.log.gz 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37327.log 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42041.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37327.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42041.log.gz 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37327.log.gz 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49662.log 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48803.log 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49662.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49662.log.gz 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48803.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48803.log.gz 2024-02-22T00:00:37.777 INFO:teuthology.orchestra.run.smithi133.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-02-22T00:00:37.778 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 92.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-02-22T00:00:38.008 INFO:teuthology.orchestra.run.smithi142.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-02-22T00:00:38.145 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/ceph-mon.b.log: file size changed while zipping 2024-02-22T00:00:38.146 INFO:teuthology.orchestra.run.smithi142.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-02-22T00:00:38.337 INFO:teuthology.orchestra.run.smithi142.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-02-22T00:00:38.510 INFO:teuthology.orchestra.run.smithi133.stderr: 89.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-02-22T00:00:38.519 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:36:06.417 36935== 2024-02-22T00:00:38.519 INFO:tasks.ceph.mon.b.smithi142.stderr:==00:00:36:06.417 36935== Exit program on first error (--exit-on-first-error=yes) 2024-02-22T00:00:38.537 INFO:tasks.ceph.mon.b.smithi142.stderr:daemon-helper: command failed with exit status 42 2024-02-22T00:00:39.565 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-02-22T00:00:39.948 INFO:teuthology.orchestra.run.smithi133.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-02-22T00:00:40.361 INFO:teuthology.orchestra.run.smithi142.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-02-22T00:00:40.961 INFO:teuthology.orchestra.run.smithi142.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-02-22T00:00:41.330 INFO:teuthology.orchestra.run.smithi142.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-02-22T00:00:41.524 INFO:teuthology.orchestra.run.smithi133.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-02-22T00:00:41.716 INFO:teuthology.orchestra.run.smithi133.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-02-22T00:00:41.853 INFO:teuthology.orchestra.run.smithi133.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-02-22T00:00:42.002 INFO:teuthology.orchestra.run.smithi142.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-02-22T00:00:44.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~5s 2024-02-22T00:00:49.494 INFO:teuthology.orchestra.run.smithi133.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.46098.log.gz 2024-02-22T00:00:49.496 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-02-22T00:00:49.496 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m11.862s 2024-02-22T00:00:49.496 INFO:teuthology.orchestra.run.smithi133.stderr:user 0m26.257s 2024-02-22T00:00:49.496 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m0.905s 2024-02-22T00:00:49.769 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~10s 2024-02-22T00:00:54.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~15s 2024-02-22T00:00:57.690 INFO:teuthology.orchestra.run.smithi142.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-02-22T00:00:57.691 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-02-22T00:00:57.691 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m20.026s 2024-02-22T00:00:57.691 INFO:teuthology.orchestra.run.smithi142.stderr:user 0m34.013s 2024-02-22T00:00:57.691 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m1.091s 2024-02-22T00:00:57.692 INFO:tasks.ceph:Archiving logs... 2024-02-22T00:00:57.692 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/remote/smithi133/log 2024-02-22T00:00:57.693 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-22T00:00:59.979 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~20s 2024-02-22T00:01:02.276 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/remote/smithi142/log 2024-02-22T00:01:02.277 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-22T00:01:05.099 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~26s 2024-02-22T00:01:07.342 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable malloc malloc strdup 2024-02-22T00:01:07.344 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-22T00:01:07.364 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bfc837d7b45ce8e095d92fa9c342964cea5e95f/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable malloc malloc strdup 2024-02-22T00:01:07.364 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-22T00:01:07.365 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-22T00:01:07.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-22T00:01:07.457 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-22T00:01:07.467 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-22T00:01:07.467 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T00:01:07.471 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T00:01:07.484 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-02-22T00:01:07.487 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-02-22T00:01:07.489 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T00:01:07.489 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-02-22T00:01:07.489 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 19 +288us[ +288us] +/- 88ms 2024-02-22T00:01:07.489 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 +109us[ +154us] +/- 57ms 2024-02-22T00:01:07.490 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -353us[ -307us] +/- 73ms 2024-02-22T00:01:07.490 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T00:01:07.493 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T00:01:07.493 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-02-22T00:01:07.493 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 19 +277us[ +284us] +/- 88ms 2024-02-22T00:01:07.493 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv02.front.sepia.ceph.com 3 6 377 16 +109us[ +116us] +/- 58ms 2024-02-22T00:01:07.493 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -358us[ -351us] +/- 73ms 2024-02-22T00:01:07.493 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T00:01:07.493 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-22T00:01:07.523 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-22T00:01:07.524 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-22T00:01:07.589 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-22T00:01:07.666 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-22T00:01:07.739 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T00:01:07.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T00:01:07.771 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T00:01:07.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T00:01:07.803 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-22T00:01:07.817 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-22T00:01:07.851 INFO:teuthology.task.internal:Duration was 3363.135738 seconds 2024-02-22T00:01:07.852 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-22T00:01:07.898 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-22T00:01:07.898 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-22T00:01:07.901 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-22T00:01:07.944 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T00:01:07.944 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T00:01:08.266 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-22T00:01:08.266 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-02-22T00:01:08.267 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-22T00:01:08.287 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-02-22T00:01:08.288 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-22T00:01:08.316 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-22T00:01:08.317 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-22T00:01:08.330 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-22T00:01:08.385 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-22T00:01:08.386 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-22T00:01:08.400 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-22T00:01:08.541 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-22T00:01:08.554 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-22T00:01:08.554 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-22T00:01:08.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-22T00:01:08.609 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-22T00:01:08.624 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-22T00:01:08.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-22T00:01:08.650 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-02-22T00:01:08.675 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-02-22T00:01:08.702 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-22T00:01:08.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T00:01:08.722 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-22T00:01:08.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T00:01:08.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-22T00:01:08.769 INFO:teuthology.task.internal:Transferring archived files... 2024-02-22T00:01:08.770 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/remote/smithi133 2024-02-22T00:01:08.770 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-22T00:01:08.811 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021/remote/smithi142 2024-02-22T00:01:08.812 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-22T00:01:08.847 INFO:teuthology.task.internal:Removing archive directory... 2024-02-22T00:01:08.848 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-22T00:01:08.850 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-22T00:01:08.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-22T00:01:08.912 INFO:teuthology.task.internal:Not uploading archives. 2024-02-22T00:01:08.913 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-22T00:01:08.950 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-22T00:01:08.978 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-22T00:01:08.980 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-22T00:01:08.993 INFO:teuthology.orchestra.run.smithi133.stdout: 265512 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 00:01 /home/ubuntu/cephtest 2024-02-22T00:01:08.994 INFO:teuthology.orchestra.run.smithi142.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 00:01 /home/ubuntu/cephtest 2024-02-22T00:01:08.996 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-22T00:01:09.035 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-22T00:01:09.092 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: valgrind error: Leak_StillReachable malloc malloc strdup 2024-02-22T00:01:09.092 INFO:teuthology.nuke:Checking targets against current locks 2024-02-22T00:01:09.117 DEBUG:teuthology.nuke:shortname: smithi133 2024-02-22T00:01:09.118 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-22T00:01:09.142 DEBUG:teuthology.nuke:shortname: smithi142 2024-02-22T00:01:09.142 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-22T00:01:09.146 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-21 22:56:29.581974', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIQO4F2tzpA2TGhD0+Q5J562sN4OU5mir5R/JLR2UmtSabM3oZhxjURS0e0nm69nVx0/HmIPnB93HuV8bXkZtqM='} 2024-02-22T00:01:09.166 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-21 22:56:29.583097', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIY6+OhPMMUUKiclL98O2Lt81V/BXxVGzuWGo/6K6ZtYxlt+DbvCAGa34duU8PucAHFMXB8TFlHfqV/kqyG528E='} 2024-02-22T00:01:09.196 INFO:teuthology.orchestra.console.smithi133:Power off 2024-02-22T00:01:09.197 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-22T00:01:09.222 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2024-02-22T00:01:09.222 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T00:01:09.241 INFO:teuthology.orchestra.console.smithi142:Power off 2024-02-22T00:01:09.241 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-22T00:01:09.251 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-02-22T00:01:09.264 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2024-02-22T00:01:09.264 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T00:01:09.289 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-02-22T00:01:10.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~31s 2024-02-22T00:01:13.252 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T00:01:13.290 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T00:01:13.364 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-02-22T00:01:13.401 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-02-22T00:01:15.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~36s 2024-02-22T00:01:15.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-02-22T00:01:17.367 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T00:01:17.404 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T00:01:17.480 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-02-22T00:01:17.516 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-02-22T00:01:20.208 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~41s 2024-02-22T00:01:20.208 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~5s 2024-02-22T00:01:21.482 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T00:01:21.518 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T00:01:21.594 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2024-02-22T00:01:21.629 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2024-02-22T00:01:21.695 INFO:teuthology.orchestra.console.smithi133:Power off completed 2024-02-22T00:01:21.730 INFO:teuthology.orchestra.console.smithi142:Power off completed 2024-02-22T00:01:21.835 INFO:teuthology.run:Summary data: description: fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} duration: 3363.1357378959656 failure_reason: 'valgrind error: Leak_StillReachable malloc malloc strdup' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-02-22T00:01:21.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-22T00:01:21.996 INFO:teuthology.run:FAIL